Image processing method, device, electronic device and storage medium

Abstract

The present invention discloses an image processing method, device, electronic device and storage medium, which specifically relate to the field of computer technology. The specific implementation scheme is: first determine the first normal infrared image corresponding to the first normal hardware in the transmission line, input the first normal infrared image into the defect generation network model, and obtain the first heating infrared image, the hardware included in the image has heating defects, and then judge the similarity between the first heating infrared image and the second heating infrared image corresponding to the first heating hardware. When the similarity between the first heating infrared image and the second heating infrared image is greater than the set threshold, determine the image sample based on the first heating infrared image, and the image sample is the sample used for training the transmission defect detection model. The purpose of expanding the image sample is achieved, the problem of scarce image samples is solved, and new technical support is provided for the training of deep learning models for image transmission defect detection.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an image processing method, device, electronic equipment and storage medium. Background Art

[0002] Transmission lines are responsible for the transmission and distribution of electric energy. The main equipment for inspection in transmission lines includes joint pipes, parallel groove clamps, crimping sleeves and other hardware.

[0003] Traditional manual transmission line inspection is not suitable for the construction and development of modern power grids due to high labor costs, dangerous working environment, and low detection efficiency. Especially when thermal defect detection of transmission lines is required, it is difficult to achieve comprehensive and timely line detection with traditional methods. Drones, with their high efficiency, flexibility, and various infrared camera equipment, are gradually replacing manual inspection as a conventional line inspection method.

[0004] However, due to the fact that there are very few infrared image samples of hardware such as joint pipes, parallel groove clamps, and crimping sleeves with heating defects in the currently smoothly operating transmission lines, the scarcity of data greatly limits the accuracy and performance of the deep learning model training of infrared image transmission defect detection. Summary of the invention

[0005] The present invention provides an image processing method, device, electronic device and storage medium, aiming to expand image samples by combining a network model with similarity discrimination to solve the current problem of small image samples and scarce data.

[0006] According to a first aspect of the present invention, there is provided an image processing method, comprising: determining a first normal infrared image corresponding to a first normal fitting in a transmission line; inputting the first normal infrared image into a defect generation network model to obtain a first heating infrared image corresponding to the first normal infrared image, wherein the fittings included in the first heating infrared image have heating defects; performing similarity judgment on the first heating infrared image and the second heating infrared image corresponding to the first heating fitting to determine the similarity between the first heating infrared image and the second heating infrared image; when the similarity is greater than a set threshold, determining an image sample based on the first heating infrared image, wherein the image sample is a sample used for training a transmission defect detection model.

[0007] According to a second aspect of the present invention, a training method for a defect generation network model is provided, comprising: for each type of sample set, selecting a second normal infrared image corresponding to a second normal hardware and a third heating infrared image corresponding to a second heating hardware from a training sample subset of the sample set; generating a fourth heating infrared image corresponding to the second normal infrared image through a first generation network in the network model to be trained; restoring the fourth heating infrared image back to a third normal infrared image through a second generation network in the network model to be trained; converting the third heating infrared image into a fourth normal infrared image through the second generation network; restoring the fourth normal infrared image to a fifth heating infrared image through the first generation network; determining a conversion loss based on the second normal infrared image, the third normal infrared image, the third normal infrared image, the third heating infrared image, the fourth heating infrared image and the fifth heating infrared image; adjusting the model parameters of the network model to be trained based on the conversion loss, and returning to continue selecting the next second normal infrared image and the next third heating infrared image from the training sample subset until a training end condition is met, and determining the trained first generation network as the defect generation network model.

[0008] According to a third aspect of the present invention, there is provided an image processing device, comprising: a first determination module, for determining a first normal infrared image corresponding to a first normal fitting in a transmission line; an input module, for inputting the first normal infrared image into a defect generation network model, to obtain a first heating infrared image corresponding to the first normal infrared image, wherein the fittings included in the first heating infrared image have heating defects; a judgment module, for performing similarity judgment between the first heating infrared image and the second heating infrared image corresponding to the first heating fitting, to determine the similarity between the first heating infrared image and the second heating infrared image; a second determination module, for determining an image sample based on the first heating infrared image when the similarity is greater than a set threshold, wherein the image sample is a sample used for training a transmission defect detection model.

[0009] According to a fourth aspect of the present invention, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image processing method described in any embodiment of the present invention.

[0010] According to a fifth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image processing method described in any embodiment of the present invention when executed.

[0011] The technical solution of the embodiment of the present invention generates an infrared image of a heating hardware fitting from an infrared image of a normal hardware fitting by utilizing a method that combines a defect generation network model and similarity judgment, thereby expanding the image samples, solving the problem of scarce image samples, and providing new technical support for the training of deep learning models for image transmission defect detection.

[0012] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0014] Figure 1 is a flowchart of an image processing method provided according to Embodiment 1 of the present invention.

[0015] Figure 2 It is a flow chart of an image processing method taking an infrared image of a splice tube as an example according to the first embodiment of the present invention.

[0016] Figure 3 It is a flow chart of a training method for a defect generation network model provided according to the second embodiment of the present invention.

[0017] Figure 4 It is a schematic diagram of a training method for a defect generation network model using an infrared image of a connecting pipe as an example provided in accordance with the second embodiment of the present invention.

[0018] Figure 5 It is a structural schematic diagram of an image processing device provided according to Embodiment 3 of the present invention.

[0019] Figure 6 It is a block diagram of an electronic device provided according to a fourth embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0022] See also Figure 1 and Figure 2 , Embodiment 1 of the present invention provides an image processing method, Figure 1 This is a flowchart of an image processing method provided according to the first embodiment of the present invention. This embodiment can be applied to the case where a normal hardware infrared image is generated into a heating hardware infrared image, thereby expanding the image sample. The method can be executed by an image processing device, which can be implemented in the form of hardware and / or software, and the image processing device can be configured in an electronic device. Figure 1 As shown, the method includes.

[0023] S110: Determine a first normal infrared image corresponding to a first normal hardware fitting in the transmission line.

[0024] In this embodiment, the transmission line can be understood as a power line equipment used for power transmission, including an overhead transmission line. The overhead transmission line is composed of line towers, conductors, insulators, hardware, guy wires, grounding devices, etc., which are erected above the ground and bear the heavy task of transmitting and distributing electric energy. Hardware can be understood as metal accessories used to connect and fix components such as conductors, lightning rods and insulators to protect conductors and insulators from mechanical damage and ensure the safe and stable operation of the line, including parallel groove clamps, insulator strings, connecting pipes, crimping sleeves, etc. The first normal hardware can be understood as a hardware that does not heat up, that is, the temperature of the hardware is within a reasonable range under normal working conditions, and there is no abnormal phenomenon of heating. Infrared images can be understood as images obtained using infrared thermal imaging technology, such as infrared radiation emitted by an object is received by an infrared detector and converted into an electrical signal, and an infrared image is formed after signal processing and image reconstruction. In infrared images, different colors represent different temperature distributions, and pseudo colors are usually used to indicate temperature changes, such as red represents high temperature, that is, a heating area. The first normal infrared image can be understood as an image obtained by processing the infrared image acquired by the first normal fitting and intercepting a specific area, such as an infrared image corresponding to the first normal fitting acquired by an infrared camera and processed. The processing is not limited here, for example, it can be an image obtained by extracting the area where the first normal fitting is located.

[0025] Specifically, the infrared camera mounted on the drone first collects infrared images of the fittings such as the joint pipe, parallel groove clamp, and crimping sleeve in the transmission line, annotates the collected infrared images, and converts the information that can identify the infrared image into a lightweight data exchange format (JavaScript Object Notation, json) format for storage, then intercepts the target area in the infrared image, preprocesses the target area, and finally forms a small sample data set with the obtained images. The small sample data set can be considered as a data set containing small samples, and the small sample can be considered as the target area after preprocessing. Since the size of the target area is smaller than the size of the infrared image collected by the infrared camera, it is called a small sample. The infrared camera collects infrared images of the fittings, which include normal fittings and heating fittings. Accordingly, the small sample data set includes small samples corresponding to normal fittings and heating fittings respectively. The target area can be understood as a specific range with a specific meaning. Different target areas can be defined according to the characteristics or attributes required by different studies. For example, the area containing a certain type of fittings can be defined as the target area. This operation can randomly select a normal infrared image from the small sample data set as the first normal infrared image, and the first normal infrared image is a normal infrared image corresponding to the first normal hardware.

[0026] For example, after the infrared images collected by the infrared camera mounted on the drone are annotated and saved, the area containing the joint pipe, parallel groove clamp, crimping sleeve and other hardware is taken as the target area, and the square enclosed area of ​​equal length and width is intercepted, and then it is scaled to n by equal proportion after preprocessing. A square image of n pixels in size is placed in a small sample data set of the corresponding type of hardware, and a first normal infrared image is selected from the small sample data set for subsequent use. In this embodiment, n is a positive integer, such as 256.

[0027] S120, inputting the first normal infrared image into a defect generation network model to obtain a first heating infrared image corresponding to the first normal infrared image, wherein the hardware included in the first heating infrared image has heating defects.

[0028] In this embodiment, the defect generation network model can be considered as a model that can generate infrared images of heating fittings. The defect generation network model is trained by infrared images of normal fittings and infrared images of heating fittings to learn the characteristics of infrared images of heating fittings. The defect generation network model consists of a generation network. The generation network is used for the mutual conversion of source domain images and target domain images, such as converting source domain images into target domain images. The difference from the traditional method is that the defect generation network model does not require paired training data, only two different image sets are needed for training, and complex conversions can be performed on the images, with certain stability and accuracy. The source domain can be understood as a data set for training the model, and the target domain can be understood as a data set for the model to be used. The first heating infrared image can be understood as an infrared image generated by the first normal infrared image through the defect generation network model, and the fittings in the target area of ​​the infrared image have been converted to a heating state, but the image does not really exist. The heating defect can be understood as an undesirable phenomenon or fault caused by abnormal increase in the temperature of the fittings due to various reasons, which may cause tripping, power outages and other faults, affecting the normal supply of electricity.

[0029] Specifically, an infrared image corresponding to a first normal hardware is randomly selected from a small sample data set as the first normal infrared image, and is input into the defect generation network model. The first heating infrared image is output through the defect generation network model. At this time, the hardware in the first heating infrared image has a heating defect. The first heating infrared image is not a heating image corresponding to a real heating hardware, but a heating image generated by the defect generation network model.

[0030] For example, Figure 2 This is a flow chart of an image processing method using an infrared image of a splice tube as an example according to the first embodiment of the present invention. Figure 2In the embodiment, the first normal infrared image 101 is first input to the input layer 102 of the defect generation network model, wherein the defect generation network model structure adopts a neural network structure 103 based on a residual convolution module, and the neural network structure based on the residual convolution module has a total of twelve layers, wherein the first layer is a convolution layer of 64 7x7 convolution kernels, with reflection filling and a step size of 1; the second and third layers are convolution layers of 256 3x3 convolution kernels and 128 3x3 convolution kernels, respectively, with a step size of 2, and each convolution layer is followed by a nonlinear activation layer; the fourth to ninth layers are all residual blocks, each residual block is provided with a jump link, and there are two 3x3 convolution layers inside the residual block; the tenth and eleventh layers are deconvolution layers containing 128 convolution kernels and 64 3x3 convolution kernels, with a step size of 1 / 2; the last layer is a convolution layer of 3 7x7 convolution kernels, with reflection filling and a step size of 1. Finally, the generated image is output to the output layer 104 of the defect generation network model and restored to 3 256 256 size, the first fever infrared image 105 can be generated.

[0031] S130: Perform similarity judgment on the first heating infrared image and the second heating infrared image corresponding to the first heating fitting to determine the similarity between the first heating infrared image and the second heating infrared image.

[0032] In this embodiment, the first heating fitting can be understood as a heating fitting, that is, the fitting is under abnormal working conditions, its temperature is higher than the reasonable range, and abnormal heating occurs. The second heating infrared image can be understood as an infrared image obtained by collecting the first heating fitting. The second heating infrared image can be the original image collected by the infrared camera, or it can be an image processed from the original image, such as an infrared image corresponding to the first heating fitting collected by the infrared camera that has been processed. When the image is collected, the fitting in the target area of ​​the first heating fitting has already had a heating defect. A heating infrared image can be randomly selected from the small sample data set as the second heating infrared image. Similarity can be understood as a generation quality metric, which can be used to evaluate the similarity between the image generated by the model and the real image. The performance of the model can be judged by this metric.

[0033] Specifically, the generated first thermal infrared image is evaluated, and the similarity between the first thermal infrared image and the second thermal infrared image is calculated to evaluate whether the generated first thermal infrared image meets the requirements. In this application, the similarity can be evaluated by the structural similarity (SSIM) index. Structural similarity is an index used to measure the similarity between two images, and the similarity is calculated by comparing the brightness, contrast and structural information of the two images in a local area.

[0034] For example, according to Figure 2 As shown, the first fever infrared image 105 and the second fever infrared image 106 are subjected to similarity detection by using the SSIM index, and the SSIM calculation formula is as follows.

[0035]

[0036] In the formula, represents the average value of structural similarity, represents the feature vector of the first fever infrared image, represents the feature vector of the second fever infrared image, represents the number of samples with heating defects included in the small sample data set corresponding to the second heating infrared image, and is the local average of the first heating infrared image and the second heating infrared image, and is the local standard deviation of the first and second heating infrared images, is the covariance between the two images, and is a constant.

[0037] S140. When the similarity is greater than a set threshold, determine an image sample based on the first heating infrared image, where the image sample is a sample used for training a power transmission defect detection model.

[0038] In this embodiment, setting a threshold can be used to determine a specific boundary or standard in order to classify the results. During the model training process, the threshold can be continuously adjusted according to the model performance to optimize the model. Image samples can be understood as image data used for analysis and model training. An image is composed of multiple pixel information, and each pixel contains a large amount of information. Image samples are the basis of image processing. By analyzing image samples, key information in the image can be extracted for use in training models. The power transmission defect detection model can be understood as a model for defect detection of infrared images in power transmission lines, which can be used to identify whether there are hardware with heating defects in infrared images.

[0039] Specifically, if the first heating infrared image is highly similar to the second heating infrared image, that is, the generated first heating infrared image is close to the real second heating infrared image, it means that the first heating infrared image is an image sample that can be used for training the power transmission defect detection model. Finally, the first heating infrared image is restored to its original size, determined as an image sample, and placed in the small sample data set to which it belongs. If the similarity between the two is low, the first heating infrared image is discarded.

[0040] For example, according to Figure 2 , input the first normal infrared image into the defect generation network model structure generation 3 256 256 size first fever infrared image, and then the first fever infrared image and 3 256 The second thermal infrared image of size 256 is subjected to similarity detection. If the similarity between the two is high, the first thermal infrared image is restored to its original size. If the similarity is low, the image is discarded.

[0041] Optionally, when the similarity is greater than a set threshold, the first thermal infrared image is restored to a set original size to obtain a restored image, and the restored image is determined as an image sample.

[0042] Specifically, if the first heating infrared image has a high similarity with the second heating infrared image, it means that the first heating infrared image is an image sample that can be used for training the power transmission defect detection model. Then, the first heating infrared image is restored to its original size, and the infrared image that has been restored to its original size is determined as an image sample and put into the corresponding small sample data set. The original size refers to the size of the infrared image captured by the infrared camera without any adjustment or processing.

[0043] The technical solution of the embodiment of the present invention generates an infrared image of a heating hardware fitting from an infrared image of a normal hardware fitting by utilizing a method combining a defect generation network model and similarity judgment, thereby achieving the purpose of expanding image samples, solving the problem of scarce image samples, and providing new technical support for the training of deep learning models for image transmission defect detection.

[0044] Based on the above embodiment, a variant embodiment of the above embodiment is proposed. It should be noted that in order to make the description concise, only the differences from the above embodiment are described in the variant embodiment.

[0045] In one embodiment, an image processing method further includes: acquiring an infrared image set containing hardware in a transmission line; intercepting a target area of ​​the infrared images included in the infrared image set, wherein the target area is an area including the hardware in the corresponding infrared image; preprocessing the target area, and adding the preprocessed image to a sample set corresponding to the type of the hardware.

[0046] In this embodiment, the infrared image set can be understood as a set of multiple infrared images, wherein the infrared images are collected by an infrared camera and all the infrared images contain hardware. The sample set can be understood as a data set containing samples, wherein the samples can be considered as the target area after preprocessing, and the sample set contains processed infrared images collected by the infrared camera, including samples corresponding to normal hardware and heating hardware.

[0047] Specifically, an infrared camera mounted on a drone is first used to collect infrared images of hardware in the transmission line. The target areas of the collected infrared images are marked and put into a set of infrared images containing hardware in the transmission line. Then, the infrared images are intercepted according to the marked target areas, and the target areas are set to the areas in the corresponding infrared images that contain hardware such as joint pipes, parallel groove clamps, and crimping sleeves. Finally, the intercepted infrared images are preprocessed by scaling, cropping, and normalization, and the preprocessed infrared images are added to the sample sets corresponding to different types of hardware.

[0048] Optionally, determining the first normal infrared image corresponding to the first normal fitting in the transmission line includes: for each type of sample set, selecting the first normal infrared image corresponding to the first normal fitting from the sample set, the first normal infrared image being a preprocessed image corresponding to the first normal fitting in the sample set, and the second heating infrared image corresponding to the first heat-generating fitting being a preprocessed image corresponding to the first heat-generating fitting in the sample set.

[0049] Specifically, a normal infrared image is randomly selected from the determined sample set as the first normal infrared image, and the first normal infrared image is the pre-processed normal infrared image corresponding to the first normal hardware. Then, a heating infrared image is randomly selected from the determined sample set as the second heating infrared image, and the second heating infrared image is the pre-processed heating infrared image corresponding to the first heating hardware.

[0050] Optionally, the target area is in a square shape, and the preprocessing of the target area includes: scaling the target area proportionally to a set pixel size.

[0051] For example, firstly, an infrared image set of the transmission line including the hardware is obtained from the acquired infrared image, then the area including the hardware in the infrared image is intercepted, and then the intercepted infrared image is preprocessed by scaling, cropping, normalization, etc., and the infrared image of the target area is scaled to n A square image of n pixels in size is finally added to the sample set corresponding to the hardware type. In this embodiment, n is a positive integer, such as 256.

[0052] See also Figure 3 and Figure 4 , Embodiment 2 of the present invention provides a method for training a defect generation network model, Figure 3 It is a flow chart of a training method for a defect generation network model provided according to the second embodiment of the present invention. This embodiment is developed for the defect generation network model in the above embodiment. The method includes:

[0053] S210 . For each type of sample set, select a second normal infrared image corresponding to a second normal hardware and a third heating infrared image corresponding to a second heating hardware from a training sample subset of the sample set.

[0054] In this embodiment, the training sample can be understood as a set of data used to train the model in the computer field. The model obtains data features by learning the training samples. During the training process, the model adjusts its own parameters to make the generation results of the training samples as close to the real results as possible. The training sample subset can be understood as a set composed of a part of the samples extracted from the training samples, which is generally random sampling. The training sample subset is used to train the model to obtain a defect generation network model. The second normal hardware can be understood as a hardware that has not heated up, that is, the hardware is in a normal working state, and its temperature is within a reasonable range, and no abnormal phenomenon occurs. It should be noted that the first normal hardware is applied to the sample generation process, and the second normal hardware is applied to the model training process, and there is no necessary connection between the two. The second heating hardware can be understood as a hardware that has heated up, that is, the hardware is in an abnormal working state, and its temperature is higher than the reasonable range, and abnormal heating occurs. It should be noted that the first heating hardware is applied to the sample generation process, and the second heating hardware is applied to the model training process, and there is no necessary connection between the two. The second normal infrared image can be understood as an image obtained by processing the infrared image collected from the second normal hardware and capturing a specific area, such as an infrared image corresponding to the second normal hardware collected by an infrared camera and processed. The processing is not limited here, such as an image obtained by extracting the area where the first normal hardware is located. The third heating infrared image can be understood as an infrared image collected from the second heating hardware. The third heating infrared image can be an image obtained by processing the original image collected by the infrared camera and capturing a specific area, such as an infrared image of the second heating hardware collected by the infrared camera that has been processed. When the image is collected, the hardware in the target area has already had a heating defect. A heating infrared image can be randomly selected from the small sample data set as the third heating infrared image.

[0055] Specifically, first, an infrared camera mounted on a drone collects infrared images of hardware such as connecting pipes, parallel groove clamps, and crimping sleeves in the transmission line, and the collected infrared images are annotated. Then, the target area in the infrared image is intercepted, and the infrared image of the intercepted area is preprocessed. Finally, the obtained infrared images are combined into a sample set, and the second normal infrared image and the third heating infrared image are selected from them.

[0056] Exemplarily, taking the infrared image of the connecting pipe as an example, first, the infrared images of the connecting pipe in the transmission line are collected, and the target area is captured and preprocessed. Then, the selected second normal infrared image and the third heating infrared image are divided into a training set (i.e., a training sample subset) and a test set (i.e., a test sample subset) in a ratio of 8:2. There are 351 samples corresponding to normal fittings in the training set, 326 samples corresponding to heating fittings in the training set, 75 samples corresponding to normal fittings in the test set, and 65 samples corresponding to heating fittings in the test set. A total of 817 connecting pipe sample data are used for model training.

[0057] S220. Generate a fourth fever infrared image corresponding to the second normal infrared image through the first generation network in the network model to be trained.

[0058] In this embodiment, the first generation network can be understood as a network model that converts a source domain image into a target domain image. The fourth fever infrared image can be understood as an infrared image generated by the second normal infrared image through the first generation network, and the hardware in the target area of ​​the infrared image has been converted to a fever state, but the image does not really exist.

[0059] Specifically, the fourth fever infrared image can be generated by inputting the second normal infrared image selected from the small sample data set into the first generation network of the network model to be trained, wherein the first generation network adopts a neural network structure based on a residual convolution module.

[0060] For example, Figure 4 is a schematic diagram of a training method for a defect generation network model using an infrared image of a splice tube as an example according to the second embodiment of the present invention. Figure 4 , the second normal infrared image 201 is input into the first generation network 202 of the network model to be trained, and then iterative training is started. Through the forward propagation algorithm, the input second normal infrared image is converted into the fourth fever infrared image 203.

[0061] S230. Restoring the fourth fever infrared image to a third normal infrared image through a second generation network in the network model to be trained.

[0062] In this embodiment, the second generation network can be understood as a network model that restores the target domain image back to the source domain image, and the third normal infrared image can be understood as an infrared image generated by the fourth fever infrared image through the second generation network.

[0063] Specifically, the fourth fever infrared image generated in S220 is input into the second generation network of the network model to be trained to generate the third normal infrared image, wherein the second generation network adopts a neural network structure based on a residual convolution module.

[0064] For example, according to Figure 4 , the fourth fever infrared image 203 is input into the second generation network 204 of the network model to be trained, and then iterative training is started. Through the forward propagation algorithm, the input fourth fever infrared image 203 is restored to the third normal infrared image 205. The forward propagation algorithm is an algorithm used to calculate the output in the model. The forward propagation algorithm starts from the input layer, passes through each hidden layer in sequence, and finally reaches the output layer. In each layer, the output of the previous layer is linearly combined with the weight of the current layer, and then the output of the current layer is obtained through an activation function. It is used as the input of the next layer to continue to propagate, and finally reaches the output layer, and the algorithm ends.

[0065] S240: Convert the third fever infrared image into a fourth normal infrared image through the second generation network.

[0066] In this embodiment, the fourth normal infrared image can be understood as an infrared image generated by the third fever infrared image through the second generation network.

[0067] Specifically, the third fever infrared image selected from the small sample data set is input into the second generation network of the network model to be trained, and the fourth normal infrared image can be generated through the neural network structure based on the residual convolution module.

[0068] For example, according to Figure 4 , the third fever infrared image 206 is input into the second generation network 204 of the network model to be trained, and then iterative training is started. Through the forward propagation algorithm, the input third fever infrared image is converted into a fourth normal infrared image.

[0069] S250: Using the first generation network, restore the fourth normal infrared image to a fifth fever infrared image.

[0070] In this embodiment, the fifth fever infrared image can be understood as an infrared image generated by the fourth normal infrared image through the first generation network.

[0071] Specifically, the fourth normal infrared image generated in S240 is input into the first generation network of the network model to be trained, and the fifth fever infrared image can be generated through the neural network structure based on the residual convolution module.

[0072] For example, according to Figure 4 , the fourth normal infrared image 207 is input into the first generation network 202 of the network model to be trained, and then iterative training is started. Through the forward propagation algorithm, the input fourth normal infrared image 207 is restored to the fifth fever infrared image 208.

[0073] S260 , determining a conversion loss based on the second normal infrared image, the third normal infrared image, the third heating infrared image, the fourth normal infrared image, the fourth heating infrared image, and the fifth heating infrared image.

[0074] In this embodiment, the conversion loss can be understood as an indicator to measure the conversion effect of the model during the image conversion process, reflecting the difference between the source domain and the target domain. By minimizing the conversion loss, the model can continuously adjust its own parameters so that the results generated by the model are closer to the target domain. At the same time, it can also more accurately measure the authenticity of the generated results. In the present invention, it is regarded as the sum of various losses.

[0075] Optionally, based on the second normal infrared image, the third normal infrared image, the third fever infrared image, the fourth normal infrared image, the fourth fever infrared image and the fifth fever infrared image, a conversion loss is determined, including: determining a first adversarial loss of a first discriminant network in the network model to be trained according to the second normal infrared image and the fourth normal infrared image, the first discriminant network being used to determine whether the fourth normal infrared image meets the source domain characteristics; determining a second adversarial loss of a second discriminant network in the network model to be trained according to the fourth fever infrared image and the third fever infrared image, the second discriminant network being used to determine whether the fourth fever infrared image meets the target domain characteristics; determining a consistency loss between the first generation network and the second generation network according to the second normal infrared image, the third normal infrared image, the third fever infrared image and the fifth fever infrared image; and determining the sum of the first adversarial loss, the second adversarial loss and the consistency loss as the conversion loss.

[0076] In this embodiment, the consistency loss can be understood as an indicator to measure the difference between the input image and the converted image, to ensure the reversibility of the conversion, and to ensure that the generative network does not lose too much key information of the image when performing image conversion. This indicator is crucial to stabilizing model training and generating effective conversion results. The discriminant network is used to determine whether the generated image is close to the real image. The first discriminant network can be understood as being responsible for distinguishing whether the image generated by the first generative network conforms to the source domain characteristics. The first adversarial loss can be understood as the confrontation between the second generative network and the first discriminant network, encouraging the generated image to be indistinguishable from the real, so as to promote the generation of more realistic images. The second discriminant network can be understood as being responsible for distinguishing whether the image generated by the second generative network conforms to the target domain characteristics. The second adversarial loss can be understood as the confrontation between the first generative network and the second discriminant network, encouraging the generated image to be indistinguishable from the real, so as to promote the generation of more realistic images.

[0077] Specifically, the first adversarial loss can be calculated based on the second normal infrared image and the fourth normal infrared image, the second adversarial loss can be calculated based on the third fever infrared image and the fourth fever infrared image, the partial consistency loss can be calculated based on the second normal infrared image and the third normal infrared image, and the partial consistency loss can be calculated based on the third fever infrared image and the fifth fever infrared image. The consistency loss of the network model to be trained can be obtained by adding the two together, and the conversion loss can be determined by adding the above three losses.

[0078] Exemplarily, the first adversarial loss may be calculated according to the second normal infrared image and the fourth normal infrared image, and the formula is as follows.

[0079]

[0080] In the formula represents the first adversarial loss, which is obtained by the second normal infrared image and the fourth normal infrared image through the first discriminant network. represents the first discriminant network, represents the source domain, Indicates the target domain.

[0081] Exemplarily, the second adversarial loss may be calculated according to the third heating infrared image and the fourth heating infrared image, and the formula is as follows.

[0082]

[0083] In the formula represents the second adversarial loss, which is obtained by the third and fourth fever infrared images through the second discriminant network. Represents the second discriminative network.

[0084] Exemplarily, the consistency loss may be calculated according to the second normal infrared image and the third normal infrared image, the third heating infrared image and the fifth heating infrared image, and the formula is as follows.

[0085]

[0086] In the formula represents the consistency loss, represents the first generation network, represents the second generation network, represents the second normal infrared image, represents the third heating infrared image, represents the third normal infrared image, It represents the loss of the process of generating the third normal infrared image from the second normal infrared image through the first generation network and the second generation network. represents the fifth fever infrared image, Represents the loss of the process in which the third fever infrared image is generated into the fifth fever infrared image through the second generation network and the first generation network.

[0087] Exemplarily, the conversion loss can be obtained by adding the consistency loss, the first adversarial loss and the second adversarial loss, and the formula is as follows.

[0088]

[0089] S270, adjusting the model parameters of the network model to be trained based on the conversion loss, and returning to continue selecting the next second normal infrared image and the next third fever infrared image from the training sample subset until the training end condition is met, and determining the trained first generation network as the defect generation network model.

[0090] In this embodiment, the model parameters can be understood as adjustable values ​​used to define the model structure. The parameters are continuously optimized during the model training process so that the model can continuously learn to achieve the purpose of training the model.

[0091] For example, Figure 4 Schematic diagram of a training method for a defect generation network model using an infrared image of a connecting tube as an example provided by the second embodiment of the present invention. First, the model parameters are initialized, and initial weights are set for the first generation network, the second generation network, the first discriminant network, and the second discriminant network. The batch size is set to 1, the step size is set to 0.002, and the exponential decay factor is set to 0.5 and 0.999. Then, the model parameters are updated according to the back propagation algorithm, and the four networks are alternately trained in multiple iterations to maintain a dynamic balance between them. When the iterative model reaches the Nash equilibrium point, that is, the loss value reaches the minimum and is stable, the parameters of the first generation network at this time are saved for subsequent use. The back propagation algorithm is an optimization algorithm for training models. The loss is back-propagated from the output layer to the input layer. By adjusting the model parameters, the output of the model is made as close to the true value as possible, thereby optimizing the model. Nash equilibrium can be understood as when the iterative model reaches the Nash equilibrium point, the parameters of the model are in a relatively stable state. At this time, the loss value usually reaches a local minimum, but this does not mean that it must be the global optimal solution. However, under the current model structure and parameter space, the model has reached a state of equilibrium.

[0092] Optionally, before determining the trained first generation network as the defect generation network model, it also includes: selecting the fifth normal infrared image corresponding to the third normal hardware from the test sample subset of the sample set; inputting the fifth normal infrared image into the trained first generation network to obtain the converted seventh heating infrared image; performing a similarity comparison between the fifth heating infrared image and the sixth heating infrared image corresponding to the third heating hardware in the test sample subset to obtain a comparison result; and determining whether to determine the trained first generation network as the defect generation network model based on the comparison result.

[0093] In this embodiment, the test sample subset can be understood as a set consisting of a part of the samples extracted from the sample set, which is generally random sampling, and can be used to quickly evaluate the model, which is helpful for timely adjustment and improvement of the model. The third normal hardware can be understood as a hardware that is not heated, that is, the temperature of the hardware is within a reasonable range under normal working conditions, and no abnormal phenomenon occurs. It should be noted that the first normal hardware is applied to the sample generation process, the second normal hardware is applied to the model training process, and the third normal hardware is applied to the model testing process, and there is no necessary connection between the three. The third heating hardware can be understood as a hardware that has heated up, that is, the temperature of the hardware is higher than the reasonable range under abnormal working conditions, and abnormal heating occurs. It should be noted that the first heating hardware is applied to the sample generation process, and the third heating hardware is applied to the model testing process, and there is no necessary connection between the three. The fifth normal infrared image can be understood as an image obtained by processing the infrared image collected by the third normal hardware and intercepting a specific area, such as an infrared image of the third normal hardware collected by an infrared camera and processed. The processing is not limited here, such as the image obtained after extracting the area where the first normal hardware is located. The sixth heating infrared image can be understood as an image obtained by processing the infrared image collected from the third heating fitting and intercepting a specific area, such as an infrared image corresponding to the third heating fitting collected by an infrared camera that has been processed. When the image is collected, the fitting in the target area has already had a heating defect. A heating infrared image can be randomly selected from the small sample data set as the sixth heating infrared image. The seventh heating infrared image can be understood as an infrared image generated by the first generation network from the fifth normal infrared image, and the fitting in the target area of ​​the infrared image has been converted to a heating state, but the image does not really exist.

[0094] Specifically, a fifth normal infrared image and a sixth fever infrared image are randomly selected from the test sample subset, and according to the method of the embodiment of the present invention, the fifth normal infrared image is input into the trained first generation network to obtain the seventh fever infrared image, and the sixth fever infrared image and the seventh fever infrared image are subjected to similarity detection. If the similarity is high, it means that the seventh fever infrared image can be used as an image sample for training a power transmission defect detection model, that is, the first generation network can be used as a defect generation network model. At this time, the parameters of the network model to be trained can be determined, and the first generation network can be determined as a defect generation network model for subsequent use.

[0095] A defect generation network model training method provided in an embodiment of the present invention describes in detail the training process of the defect generation network model. By selecting infrared images from a sample set and repeatedly adjusting parameters for training, a defect generation network model suitable for the present invention is finally obtained. This model can be used to achieve the purpose of expanding image samples and solve the problem of scarce image samples.

[0096] In the training process of the defect generation network model, S210 is executed first, then S220 and S240 can be executed synchronously, and S220 and S230 are executed sequentially, and S240 and S250 are also executed sequentially. After the above steps are completed, S260 and S270 are finally executed sequentially to complete the training of the network model.

[0097] See also Figure 5 , Embodiment 3 of the present invention provides an image processing device, Figure 5 FIG. 1 is a schematic diagram of the structure of an image processing device provided in Embodiment 3 of the present invention. Figure 5 As shown, the device includes.

[0098] The first determining module 310 is used to determine a first normal infrared image corresponding to a first normal hardware in the transmission line.

[0099] The input module 320 is used to input the first normal infrared image into the defect generation network model to obtain a first heating infrared image corresponding to the first normal infrared image, and the hardware included in the first heating infrared image has heating defects.

[0100] The judgment module 330 is used to perform similarity judgment on the first heating infrared image and the second heating infrared image corresponding to the first heating fitting, and determine the similarity between the first heating infrared image and the second heating infrared image.

[0101] The second determination module 340 is used to determine an image sample based on the first heating infrared image when the similarity is greater than a set threshold, and the image sample is a sample used for training the power transmission defect detection model.

[0102] The technical solution of the embodiment of the present invention completes the image processing through the mutual cooperation and collaboration between modules, and generates infrared image samples of heating hardware from infrared images of normal hardware by utilizing the defect generation network model and the similarity judgment method, thereby expanding the image samples and solving the problem of scarce image samples. It provides new technical support for the training of deep learning models for image transmission defect detection.

[0103] In one embodiment, the second determination module 340 is specifically used to: when the similarity is greater than a set threshold, restore the first thermal infrared image to a set original size to obtain a restored image; and determine the restored image as an image sample.

[0104] In one embodiment, the image processing device further includes a sample set determination module, including: an acquisition unit, used to acquire an infrared image set containing hardware in the transmission line; a capture unit, used to capture a target area of ​​the infrared image included in the infrared image set, wherein the target area is an area including the hardware in the corresponding infrared image; and a preprocessing unit, used to preprocess the target area and add the preprocessed image to the sample set corresponding to the type of the hardware.

[0105] In one embodiment, the first determination module 310 is specifically used to: for each type of sample set, select a first normal infrared image corresponding to a first normal hardware in the sample set, wherein the first normal infrared image is a preprocessed image corresponding to the first normal hardware in the sample set, and the second heating infrared image corresponding to the first heating hardware is a preprocessed image corresponding to the first heating hardware in the sample set.

[0106] In one embodiment, the target area is in a square shape, and the preprocessing unit is specifically configured to: proportionally scale the target area to a set pixel size.

[0107] In one embodiment, the defect generation network model includes a cyclic adversarial generation network model, and the image processing device also includes a training module, including: a selection unit, for selecting, for each type of sample set, a second normal infrared image corresponding to the second normal hardware and a third heating infrared image corresponding to the second heating hardware from a training sample subset of the sample set; a generation unit, for generating a fourth heating infrared image corresponding to the second normal infrared image through a first generation network in the network model to be trained; a first restoration unit, for restoring the fourth heating infrared image back to the third normal infrared image through a second generation network in the network model to be trained; a conversion unit, for converting the third heating infrared image into the third normal infrared image through the second generation network. The external image is converted into a fourth normal infrared image; a second restoration unit is used to restore the fourth normal infrared image to a fifth heating infrared image through the first generation network; a conversion loss determination unit is used to determine the conversion loss based on the second normal infrared image, the third normal infrared image, the third heating infrared image, the fourth normal infrared image, the fourth heating infrared image and the fifth heating infrared image; a network determination unit is used to adjust the model parameters of the network model to be trained based on the conversion loss, and return to continue selecting the next second normal infrared image and the next third heating infrared image from the training sample subset until the training end condition is met, and the trained first generation network is determined as the defect generation network model.

[0108] In one embodiment, the image processing device also includes a verification module, which is specifically used to: select a fifth normal infrared image corresponding to the third normal hardware in the test sample subset of the sample set; input the fifth normal infrared image to the trained first generation network to obtain a converted seventh heating infrared image; perform a similarity comparison between the fifth heating infrared image and the sixth heating infrared image corresponding to the third heating hardware in the test sample subset to obtain a comparison result; and determine whether to determine the trained first generation network as a defect generation network model based on the comparison result.

[0109] In one embodiment, the conversion loss determination unit is specifically used to: determine the first adversarial loss of the first discriminant network in the network model to be trained according to the second normal infrared image and the fourth normal infrared image, wherein the first discriminant network is used to determine whether the fourth normal infrared image conforms to the source domain characteristics; determine the second adversarial loss of the second discriminant network in the network model to be trained according to the fourth fever infrared image and the third fever infrared image, wherein the second discriminant network is used to determine whether the fourth fever infrared image conforms to the target domain characteristics; determine the consistency loss between the first generation network and the second generation network according to the second normal infrared image, the third normal infrared image, the third fever infrared image and the fifth fever infrared image; and determine the sum of the first adversarial loss, the second adversarial loss and the consistency loss as the conversion loss.

[0110] See also Figure 6 , Embodiment 4 of the present invention provides an electronic device and a computer-readable storage medium.

[0111] Figure 6 is a block diagram of an electronic device provided according to Embodiment 4 of the present invention, which can implement the image processing method described in the embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0112] like Figure 6 As shown, the electronic device 410 includes at least one processor 411, and a memory connected to the at least one processor 411 in communication, such as a read-only memory (ROM) 412, a random access memory (RAM) 413, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 411 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 412 or the computer program loaded from the storage unit 418 to the random access memory (RAM) 413. In the RAM 413, various programs and data required for the operation of the electronic device 410 can also be stored. The processor 411, the ROM 412, and the RAM 413 are connected to each other via a bus 414. An input / output (I / O) interface 415 is also connected to the bus 414.

[0113] A number of components in the electronic device are connected to the I / O interface 415, including: an input unit 416, such as a keyboard, a mouse, etc.; an output unit 417, such as various types of displays, speakers, etc.; a storage unit 418, such as a disk, an optical disk, etc.; and a communication unit 419, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 419 allows the electronic device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0114] The processor 411 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 411 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 411 performs the various methods and processes described above, such as an image processing method.

[0115] In some embodiments, the image processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 418. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 410 via the ROM 412 and / or the communication unit 419. When the computer program is loaded into the RAM 413 and executed by the processor 411, one or more steps of the image processing method described above may be performed. Alternatively, in other embodiments, the processor 411 may be configured to perform the image processing method in any other appropriate manner (e.g., by means of firmware).

[0116] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0117] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0118] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0119] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0120] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0121] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0122] According to the technical solution of the embodiment of the present invention, through an image processing method, device, electronic device and storage medium, the image samples are expanded, the problem of scarcity of image samples is solved, and new technical support is provided for the training of deep learning models for image transmission defect detection.

[0123] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0124] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An image processing method, characterized in that: include: Determine a first normal infrared image corresponding to a first normal hardware in the transmission line; The first normal infrared image is input into the defect generation network model to obtain a first heating infrared image corresponding to the first normal infrared image, wherein the hardware included in the first heating infrared image has heating defects, the defect generation network model is composed of a generation network, the defect generation network model includes a cyclic adversarial generation network model, and the training operation of the defect generation network model includes: For each type of sample set, selecting a second normal infrared image corresponding to the second normal hardware and a third heating infrared image corresponding to the second heating hardware from a training sample subset of the sample set; Generate a fourth fever infrared image corresponding to the second normal infrared image by using a first generation network in the network model to be trained; Restoring the fourth fever infrared image to a third normal infrared image through a second generation network in the network model to be trained; converting the third fever infrared image into a fourth normal infrared image through the second generation network; Restoring the fourth normal infrared image to a fifth fever infrared image through the first generating network; determining a conversion loss based on the second normal infrared image, the third normal infrared image, the third heating infrared image, the fourth normal infrared image, the fourth heating infrared image, and the fifth heating infrared image; Adjusting the model parameters of the network model to be trained based on the conversion loss, returning to continue selecting the next second normal infrared image and the next third fever infrared image from the training sample subset until the training end condition is met, and determining the trained first generation network as the defect generation network model; Performing similarity judgment on the first heating infrared image and the second heating infrared image corresponding to the first heating hardware to determine the similarity between the first heating infrared image and the second heating infrared image; When the similarity is greater than a set threshold, an image sample is determined based on the first heating infrared image, and the image sample is a sample used for training a power transmission defect detection model.

2. The image processing method according to claim 1, characterized in that: When the similarity is greater than a set threshold, determining an image sample based on the first fever infrared image includes: When the similarity is greater than a set threshold, restoring the first heating infrared image to a set original size to obtain a restored image; The restored image is determined as an image sample.

3. The image processing method according to claim 1, characterized in that: Also includes: Obtain infrared image sets containing hardware on the transmission line; Intercepting a target area of ​​the infrared image included in the infrared image set, the target area being an area including hardware in the corresponding infrared image; The target area is preprocessed, and the preprocessed image is added to a sample set corresponding to the type of the hardware.

4. The image processing method according to claim 3, characterized in that: The step of determining a first normal infrared image corresponding to a first normal hardware fitting in the power transmission line includes: For each type of sample set, a first normal infrared image corresponding to a first normal hardware is selected from the sample set, the first normal infrared image is a preprocessed image corresponding to the first normal hardware in the sample set, and the second heating infrared image corresponding to the first heating hardware is a preprocessed image corresponding to the first heating hardware in the sample set.

5. The image processing method according to claim 3, characterized in that: The shape of the target area is a square, and the preprocessing of the target area includes: The target area is proportionally scaled to a set pixel size.

6. The image processing method according to claim 1, characterized in that: Before determining the trained first generation network as the defect generation network model, the method further includes: Selecting a fifth normal infrared image corresponding to the third normal hardware from the test sample subset of the sample set; Inputting the fifth normal infrared image into the trained first generation network to obtain a converted seventh fever infrared image; Performing a similarity comparison between the fifth heating infrared image and the sixth heating infrared image corresponding to the third heating fitting in the test sample subset to obtain a comparison result; Based on the comparison result, it is determined whether to determine the trained first generation network as a defect generation network model.

7. The image processing method according to claim 1, characterized in that: The determining of the conversion loss based on the second normal infrared image, the third normal infrared image, the third heating infrared image, the fourth normal infrared image, the fourth heating infrared image, and the fifth heating infrared image comprises: Determining a first adversarial loss of a first discriminant network in the network model to be trained according to the second normal infrared image and the fourth normal infrared image, wherein the first discriminant network is used to determine whether the fourth normal infrared image meets source domain features; Determine, according to the fourth fever infrared image and the third fever infrared image, a second adversarial loss of a second discriminant network in the network model to be trained, wherein the second discriminant network is used to determine whether the fourth fever infrared image meets the target domain characteristics; determining a consistency loss between the first generation network and the second generation network according to the second normal infrared image, the third normal infrared image, the third fever infrared image, and the fifth fever infrared image; A sum of the first adversarial loss, the second adversarial loss, and the consistency loss is determined as a conversion loss.

8. An image processing device, characterized in that: include: A first determining module, used to determine a first normal infrared image corresponding to a first normal hardware in the transmission line; The input module is used to input the first normal infrared image into the defect generation network model to obtain a first heating infrared image corresponding to the first normal infrared image, wherein the hardware included in the first heating infrared image has heating defects, the defect generation network model is composed of a generation network, the defect generation network model includes a cyclic adversarial generation network model, and the training operation of the defect generation network model includes: A selection module, configured to select, for each type of sample set, a second normal infrared image corresponding to the second normal hardware and a third heating infrared image corresponding to the second heating hardware from a training sample subset of the sample set; A generating module, used for generating a fourth fever infrared image corresponding to the second normal infrared image by using a first generating network in the network model to be trained; A first restoration module, used for restoring the fourth fever infrared image back to a third normal infrared image through a second generation network in the network model to be trained; a conversion module, configured to convert the third heating infrared image into a fourth normal infrared image through the second generation network; a second restoration module, configured to restore the fourth normal infrared image into a fifth heating infrared image through the first generation network; a loss determination module, configured to determine a conversion loss based on the second normal infrared image, the third normal infrared image, the third heating infrared image, the fourth normal infrared image, the fourth heating infrared image, and the fifth heating infrared image; A model determination module, used for adjusting the model parameters of the network model to be trained based on the conversion loss, and returning to continue selecting the next second normal infrared image and the next third fever infrared image from the training sample subset until the training end condition is met, and determining the trained first generation network as the defect generation network model; A judgment module, used for judging the similarity between the first heating infrared image and the second heating infrared image corresponding to the first heating fitting, and determining the similarity between the first heating infrared image and the second heating infrared image; The second determination module is used to determine an image sample based on the first heating infrared image when the similarity is greater than a set threshold, and the image sample is a sample used for training a power transmission defect detection model.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image processing method according to any one of claims 1 to 7 when executed.

Images