Image processing method, image processing device and electronic equipment

Abstract

The invention discloses an image processing method, an image processing device and electronic equipment, and relates to the field of image processing. The definition of a target in an image can be improved. The image processing method comprises the following steps: acquiring an infrared image and a visible light image of a power transmission line in an inspection area; performing target detection on the visible light image to obtain a first area where the power transmission line is located in the visible light image, and performing target detection on the infrared image to obtain a first infrared area where the power transmission line is located in the infrared image; performing target detection on a first area in the visible light image to obtain a second area where a detection target is located, and performing target detection on a first infrared area in the infrared image to obtain a second infrared area where the insulator and the wire clamp are located; the detection target comprises an insulator and a wire clamp; determining a target area where the insulator and the wire clamp are located in combination with the second area and the second infrared area; and fusing the target area of the infrared image and the target area of the visible light image to obtain a target image.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image processing method, an image processing device and an electronic device. Background Art

[0002] Due to the convenience of drone inspections, drones are often used in power systems to collect images on transmission lines for potential hazards analysis to ensure the normal use of transmission lines. The background of images collected by drones may be very complex, including different objects, and the images collected by drones are also easily affected by weather and environmental factors, such as strong winds, heavy fog, rain, etc., resulting in low image clarity, which is not conducive to subsequent target detection and recognition tasks. Summary of the invention

[0003] The present application provides an image processing method, an image processing device and an electronic device, which can improve the clarity of an object in an image.

[0004] In a first aspect, the present application provides an image processing method, comprising:

[0005] Obtain infrared images and visible light images of the power transmission lines in the inspection area;

[0006] Performing target detection on the visible light image to obtain a first area where the power transmission line is located in the visible light image, and performing target detection on the infrared image to obtain a first infrared area where the power transmission line is located in the infrared image;

[0007] Performing target detection on the first area in the visible light image to obtain a second area where the detection target is located, and performing target detection on the first infrared area in the infrared image to obtain a second infrared area where an insulator and a wire clamp are located;

[0008] Wherein, the detection targets include insulators and wire clamps;

[0009] Determine the target area where the insulator and the wire clamp are located by combining the second area and the second infrared area;

[0010] The target area of ​​the infrared image is fused with the target area of ​​the visible light image to obtain a target image.

[0011] According to the image processing method provided by the embodiment of the present application, on the one hand, by jointly performing target detection on infrared images and visible light images, the position of the target can be corrected and the accuracy of target detection can be improved. On the other hand, target detection is first performed on the image to extract the area where the transmission line is located, and then target detection is performed in the area to extract the area where the insulators and wire clamps are located. After each target detection, the area for the next target detection can be reduced, thereby removing the influence of some objects. Through multiple target detections, the influence of other objects in the image can be minimized, thereby further improving the accuracy of target detection. On the other hand, by fusing the target area of ​​the infrared image with the target area of ​​the visible light image, the clarity of the insulators and wire clamps in the target area can be enhanced, the target can be highlighted, the status of the insulators and wire clamps can be observed, and the subsequent identification of the insulators and wire clamps is beneficial.

[0012] In a second aspect, the present application provides an image processing device, comprising:

[0013] An image acquisition module is used to acquire infrared images and visible light images of the power transmission lines in the inspection area;

[0014] A first detection module is used to perform target detection on the visible light image to obtain a first area where the power transmission line is located in the visible light image, and to perform target detection on the infrared image to obtain a first infrared area where the power transmission line is located in the infrared image;

[0015] A second detection module is used to perform target detection on the first area in the visible light image to obtain a second area where the detection target is located, and to perform target detection on the first infrared area in the infrared image to obtain a second infrared area where the insulator and the wire clamp are located; wherein the detection target includes the insulator and the wire clamp;

[0016] A target positioning module, used for determining a target area where the insulator and the wire clamp are located in combination with the second area and the second infrared area;

[0017] The image fusion module is used to fuse the target area of ​​the infrared image with the target area of ​​the visible light image to obtain a target image.

[0018] In a third aspect, the present application provides an electronic device, the electronic device comprising a memory and one or more processors. The memory stores one or more computer programs, the computer programs comprising instructions, and when the instructions are executed by the processor, the electronic device can perform the image processing method in the first aspect.

[0019] In a fourth aspect, the present application provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on an electronic device, the electronic device executes the image processing method in the first aspect.

[0020] In a fifth aspect, the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the image processing method as described in the first aspect.

[0021] It can be understood that the beneficial effects that can be achieved by the above-mentioned image processing device, electronic device, computer-readable storage medium, and computer program product can refer to the beneficial effects in the first aspect and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A flowchart of an image processing method provided in an embodiment of the present application;

[0023] Figure 2 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0024] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first", "second" and the like are used to distinguish the same items or similar items with substantially the same functions and effects. For example, the first chip and the second chip are only used to distinguish different chips, and their order is not limited. Those skilled in the art can understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not necessarily limit the difference. It should be noted that in the embodiments of the present application, the words "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or design solutions. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way. In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more.

[0026] It should be noted that the “at…” in the embodiments of the present application may be the instant when a certain situation occurs, or may be a period of time after the occurrence of a certain situation, and the embodiments of the present application do not specifically limit this.

[0027] The implementation of this embodiment will be described in detail below with reference to the accompanying drawings.

[0028] This embodiment provides an image processing method. By way of example, the image processing method can be applied to various electronic devices such as computers (PCs), tablet computers, virtual reality / augmented reality devices, wearable devices, industrial computers, and vehicle computers; it can also be applied to servers, cloud computing, server clusters, etc., and this embodiment does not make any special limitations on this.

[0029] Figure 1 A schematic flow chart of an image processing method provided in an embodiment of the present application is shown.

[0030] like Figure 1 As shown, the image processing method may include the following steps:

[0031] Step 101: Acquire infrared images and visible light images of power transmission lines in an inspection area.

[0032] By photographing the power transmission lines in the inspection area with infrared cameras, thermal imagers, etc., infrared images can be obtained. Visible light images refer to images obtained with ordinary cameras, such as RGB images.

[0033] This embodiment can be applied to drone inspections. An infrared camera and a camera are set on the drone to obtain infrared images and visible light images of the transmission line at each location in the inspection area, and the location where each image is obtained can be recorded.

[0034] This embodiment also includes: acquiring an infrared image and a visible light image of the power transmission line in the inspection area; extracting the maximum value and the minimum value of the pixel points in the infrared image; calculating the stretching ratio through the maximum value and the minimum value, and updating the pixel value of each pixel point in the infrared image based on the stretching ratio to obtain an updated infrared image; and filtering the visible light image to obtain a processed visible light image.

[0035] After obtaining the infrared image and the visible light image through the camera, the infrared image and the visible light image can be preprocessed respectively to reduce the noise in the image and improve the signal-to-noise ratio.

[0036] For infrared images, the contrast of the image is improved by stretching the image. The stretching ratio is:

[0037] X=(x max -x min ) / (max-min) (1)

[0038] Among them, max is the maximum value of the pixel value in the infrared image, and min is the minimum value of the pixel value in the infrared image. max and x min is the grayscale range, set according to actual needs, for example, xmax The value of can be 1 to 1.5 times of max, x min The value can be 0.5 to 1 times of min.

[0039] After calculating the stretch ratio, recalculate the pixel value of the original pixel point according to the stretch ratio. The formula is as follows:

[0040] X new =x min +X(x-min) (2)

[0041] Among them, X new Where is the updated pixel value of the pixel, and x is the original pixel value of the pixel.

[0042] The pixel value of each pixel in the infrared image is calculated by the above formula, and then the original pixel value is replaced to obtain the updated infrared image. Since the grayscale difference of the infrared image is relatively small, the original pixel value is increased by stretching the ratio, which can enhance the contrast of the infrared image.

[0043] For visible light images, different intensities of visible light from the outside world will cause noise to the visible light images. By filtering the visible light images, stray light in the images can be removed to improve the image quality. The filtering algorithms may include mean filtering, median filtering, Gaussian filtering or bilateral filtering. After filtering the visible light images through the filtering algorithms, a new visible light image is obtained, and subsequent operations are performed based on the new visible light image.

[0044] Step 102: performing target detection on the visible light image to obtain a first region where the power transmission line is located in the visible light image, and performing target detection on the infrared image to obtain a first infrared region where the power transmission line is located in the infrared image.

[0045] By using a target detection algorithm to detect targets in visible light images and infrared images, the position of the target, i.e., the transmission line, in the image can be determined. The target detection algorithm locates the area of ​​the target object by extracting features from the image, such as the edge, texture, color, etc. of the object, thereby determining the category and bounding box of the target object. Exemplarily, the target detection algorithm may include various models such as R-CNN, YOLO, and SSD.

[0046] The area where the power transmission line is located determined by performing target detection on the visible light image is the first area, and the area where the power transmission line is located determined by performing target detection on the infrared image is the first infrared area.

[0047] Visible light images and infrared images can be subjected to the same target detection processing. This embodiment uses visible light images as an example to illustrate the specific process of target detection processing. Exemplarily, performing target detection on the visible light image to obtain the first area where the power transmission line is located in the visible light image specifically includes: calculating a binarization threshold based on each pixel point of the visible light image, binarizing the visible light image using the binarization threshold to obtain a first image; and determining the first area where the power transmission line is located from the first image.

[0048] The process of binarization is: calculate the average value of all pixels in the image as the binarization threshold, binarize the pixels in the image according to the binarization threshold, for example, pixels below the binarization threshold are black, and pixels above the binarization threshold are white. The processed image is the first image. The white area is the transmission line, and the boundary of the first area can be determined according to the position coordinates of the white area in the first image. The first area can be represented by the coordinates of its boundary points.

[0049] Furthermore, in order to improve the accuracy of binarization, the image can be cut into 10 blocks on average, and the average grayscale value of all pixels in each block is calculated as the threshold of binarization. According to this threshold, the block is binarized, that is, the value below the threshold is black, and the value above the threshold is white, and so on until all 10 blocks are binarized, thereby obtaining the first image. Similarly, the same binarization process is performed on the infrared image to obtain the first infrared image corresponding to the infrared image, and then the first infrared region where the power transmission line is located is determined from the first infrared image. By binarizing the infrared image and the visible light image respectively, the background and the power transmission line can be distinguished, thereby obtaining the area where the power transmission line is located, that is, the first area and the first infrared area.

[0050] Step 103: Perform target detection on the first area in the visible light image to obtain a second area where the detection target is located, and perform target detection on the first infrared area in the infrared image to obtain a second infrared area where insulators and wire clamps are located. The detection targets include insulators and wire clamps.

[0051] In step 102, the area where the transmission line is located is determined through target detection. In step 103, based on the area detected in step 102, target detection is performed again to narrow the detection range, refine the target, and further detect the area where the insulator and the wire clamp are located. It can be understood that this embodiment can also perform target detection three, four or other times to ultimately accurately target and accurately position. For example, this embodiment can perform target detection on the basis of the first area to obtain the second area where the insulator is located, and then perform a third target detection on the second area to obtain the third area where the wire clamp is located. According to needs, any one or all of the second area and the third area can be used as the final target area.

[0052] The following continues to use the visible light image as an example to explain the process of determining the area where the insulator and the wire clamp are located. Specifically, performing target detection on the first area in the visible light image to obtain the second area where the detection target is located includes: calculating a binarization threshold based on the pixel points in the first area in the visible light image, binarizing the first area using the binarization threshold to obtain a second image; and determining the second area where the detection target is located from the second image.

[0053] According to the position of the first area, the average value of the pixels in the first area in the visible light image is calculated to obtain the binarization threshold of this binarization, and then the first area is binarized using the binarization threshold. Alternatively, the first area can be evenly divided into multiple blocks, and a binarization threshold is calculated for each block to binarize the block, which can make the binarization threshold more accurate, thereby improving the binarization effect. After binarization, a second image can be obtained, in which the black area or the white area is the area where the target (insulator and wire clamp) is located. In actual applications, the color of the area can be selected by the user.

[0054] The infrared image is processed as above to obtain the second infrared region where the insulator and the wire clamp are located, and the visible light image is processed as above to obtain the second region where the insulator and the wire clamp are located.

[0055] Step 104: Determine a target area where the insulator and the wire clamp are located by combining the second area and the second infrared area.

[0056] Exemplarily, the region may include multiple boundary points, and the average value of the position coordinates of the boundary points included in the second region and the position coordinates of the boundary points included in the second infrared region is calculated to obtain the target region, or the second region and the second infrared region may be weighted with different weights according to actual needs or experience, and then the average value of the weighted summation result is calculated as the value of the boundary point of the target region. Exemplarily, the weight of the region determined based on the infrared image may be greater than the weight of the region determined based on the visible light image. Alternatively, one of the second region and the second infrared region is used as the final target region.

[0057] In this embodiment, the area where the insulators and wire clamps are located is detected by infrared images, and the area where the insulators and wire clamps are located is detected by visible light images. The two images are calculated from two accuracy dimensions, and the final result is determined by combining the results of the two calculations. The target detection in the visible light image dimension and the target detection in the infrared image dimension can be corrected to each other, thereby improving the accuracy of target positioning.

[0058] This embodiment also includes: combining the first area with the first infrared area to determine the detection area where the transmission line is located; performing target detection on the detection area in the visible light image to obtain a second area where insulators and wire clamps are located; and performing target detection on the detection area in the infrared image to obtain a second infrared area where insulators and wire clamps are located.

[0059] That is to say, after the first target detection, the areas detected in the two images are corrected to further reduce the error of target detection and improve accuracy.

[0060] Step 105: Fusing the target area of ​​the infrared image with the target area of ​​the visible light image to obtain a target image.

[0061] Image fusion refers to the process of extracting the useful information from each channel to the maximum extent through image processing and computer technology from image data of the same target collected by multiple source channels, and finally integrating them into high-quality images to improve the utilization rate of image information, improve the accuracy and reliability of computer interpretation, and enhance the spatial resolution and spectral resolution of the original image, which is beneficial for monitoring.

[0062] By fusing the target area of ​​the infrared image with the target area of ​​the visible light image, the target area can be enhanced and highlighted, making it easier for users to observe insulators and wire clamps. For pixels outside the target area, the pixel value or gray value average can be used to fill in as the background, thereby reducing image noise and improving the clarity of the target in the image, which is beneficial for subsequent image recognition or other processing.

[0063] Specifically, the average value of the pixels in the target area of ​​the infrared image and the pixels in the target area of ​​the visible light image is calculated, and the result is used as the pixel value of the corresponding pixel in the target image, or the value of the pixel in the target area of ​​the infrared image is used as depth information, which is converted into the RGB color space to obtain the corresponding value, and then added to the value of the same pixel in the visible light image to obtain the final target image.

[0064] Exemplarily, the pixels in the target area in the visible light image and the pixels in the target area in the infrared image are weighted averaged to obtain a fused target image. The weights of the pixels in the infrared image and the weights of the pixels in the visible light image can be adjusted according to actual needs.

[0065] Further, the weighted averaged pixels are converted to the HIS color space to obtain the target image. The HIS color space is a model based on visual perception for describing colors. HIS stands for hue, saturation, and intensity. This model is different from the RGB model. RGB focuses more on hardware implementation and color display, while HIS is closer to the human eye's perception of color. The conversion between these two models can be achieved through different mathematical models, such as cylindrical coordinate models, triangle models, triangular pyramid coordinate models, etc.

[0066] In this embodiment, other fusion methods may be used to fuse the target area of ​​the infrared image with the target area of ​​the visible light image, such as through PCA transformation, pyramid fusion, wavelet transform fusion, deep learning model fusion, etc., but this embodiment is not limited thereto.

[0067] The PCA transformation method projects the target area in the infrared image and the visible light image into a new coordinate system through a linear transformation. In this new coordinate system, the first coordinate axis corresponds to the direction with the largest data variance (i.e., the first principal component), the second coordinate axis corresponds to the direction with the second largest variance (i.e., the second principal component), and so on. In this way, the pixels are independent of each other in each dimension in the new coordinate system, and the amount of information in each dimension decreases; after fusion, a new image of the target area is obtained, i.e., the target image.

[0068] For other pixels outside the target area of ​​the infrared image and the visible light image, they are fused as the background part according to the method of calculating the average grayscale, and fused with the target image obtained in the above steps to form a new image. The size of this image can be consistent with that of the original visible light image and infrared image.

[0069] This embodiment also includes: using the target image to identify the current use status of the insulator and the wire clamp; when there is an abnormality in the use status, obtaining position information corresponding to the target image.

[0070] The usage status can be used to indicate whether the insulator and the wire clamp are normal or abnormal, as well as the specific abnormality type, such as temperature fault, ice and snow coverage, damage, etc. Exemplarily, the target image is input into the deep learning model, and the model is used to identify the usage status of the insulator and the wire clamp in the target image. After training, the deep learning model can output whether the usage status of the insulator and the wire clamp is normal or abnormal, as well as the specific abnormality type. When an abnormality is identified, the original image corresponding to the target image, that is, the acquisition position of the visible light image and the infrared image, can be obtained as the location information of the abnormality.

[0071] It can be understood that other information of insulators and wire clamps, such as type, etc., can also be identified through the deep learning model.

[0072] For example, the infrared image corresponding to the target image and the target image can be used together as the input of the deep learning model to identify whether the use status of the insulator and the wire clamp is normal. The infrared image also includes the temperature information of the object, so inputting the infrared image and the target image together into the deep learning model is conducive to the deep learning model to extract features and improve the recognition accuracy of the use status of the insulator and the wire clamp.

[0073] This embodiment completes the fusion of infrared light images and visible light images by preprocessing the infrared light images, preprocessing the visible light images, extracting feature areas, fusing feature areas, synthesizing new images, etc., thereby highlighting the feature areas and reducing the impact of environmental and weather factors on the inspection images.

[0074] Furthermore, this embodiment also provides an image processing device, which can be used to execute the above-mentioned image processing method. Figure 2 As shown, the image processing device 200 specifically includes: an image acquisition module 201, which is used to acquire an infrared image and a visible light image of the power transmission line in the inspection area; a first detection module 202, which is used to perform target detection on the visible light image to obtain a first area where the power transmission line is located in the visible light image, and to perform target detection on the infrared image to obtain a first infrared area where the power transmission line is located in the infrared image; a second detection module 203, which is used to perform target detection on the first area in the visible light image to obtain a second area where the detection target is located, and to perform target detection on the first infrared area in the infrared image to obtain a second infrared area where insulators and wire clamps are located; wherein the detection target includes insulators and wire clamps; a target positioning module 204, which is used to determine the target area where the insulators and wire clamps are located in combination with the second area and the second infrared area; an image fusion module 205, which is used to fuse the target area of ​​the infrared image with the target area of ​​the visible light image to obtain a target image.

[0075] In an exemplary embodiment, the image processing device 200 also includes: an area correction module, which is used to determine the detection area where the transmission line is located by combining the first area and the first infrared area; perform target detection on the detection area in the visible light image to obtain a second area where the insulators and wire clamps are located; and perform target detection on the detection area in the infrared image to obtain a second infrared area where the insulators and wire clamps are located.

[0076] In an exemplary embodiment, the first detection module 202 is specifically used to: calculate a binarization threshold based on each pixel point of the visible light image, binarize the visible light image using the binarization threshold to obtain a first image; and determine a first area where a power transmission line is located from the first image.

[0077] In an exemplary embodiment, the second detection module 203 is specifically used to: calculate a binarization threshold based on pixel points in the first area in the visible light image, binarize the first area using the binarization threshold to obtain a second image; and determine a second area where the detection target is located from the second image.

[0078] In an exemplary embodiment, the image processing device 200 further includes a target recognition module for using the target image to recognize the current use status of the insulator and the wire clamp; when there is an abnormality in the use status, obtaining position information corresponding to the target image.

[0079] In an exemplary embodiment, the image fusion module 205 is specifically used to: perform weighted averaging of the pixel points in the target area in the visible light image and the pixel points in the target area in the infrared image to obtain a fused target image.

[0080] In an exemplary implementation, the image fusion module 205 is specifically configured to: convert the weighted averaged pixel points into the HIS color space to obtain a target image.

[0081] In an exemplary embodiment, the image acquisition module 201 is specifically used to: acquire infrared images and visible light images of the power transmission lines in the inspection area; extract the maximum and minimum values ​​of the pixels in the infrared image; calculate the stretching ratio through the maximum and minimum values, and update the pixel value of each pixel in the infrared image based on the stretching ratio to obtain an updated infrared image; and filter the visible light image to obtain a processed visible light image.

[0082] The specific details of each module or unit in the above-mentioned image processing device have been described in detail in the corresponding image processing method, so they will not be repeated here.

[0083] The embodiment of the present application also provides an electronic device, Figure 3 A schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure is shown. Figure 3 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0084] like Figure 3 As shown, electronic device 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. In RAM 603, various programs and data required for system operation are also stored. CPU 601, ROM 602 and RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to bus 604.

[0085] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0086] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above-mentioned functions defined in the embodiment of the present application are executed.

[0087] For example, when the computer program is executed by the central processing unit (CPU) 601, the following can be performed: acquiring an infrared image and a visible light image of the transmission line in the inspection area; performing target detection on the visible light image to obtain a first area where the transmission line is located in the visible light image, and performing target detection on the infrared image to obtain a first infrared area where the transmission line is located in the infrared image; performing target detection on the first area in the visible light image to obtain a second area where the detection target is located, and performing target detection on the first infrared area in the infrared image to obtain a second infrared area where insulators and wire clamps are located; wherein the detection target includes insulators and wire clamps; determining the target area where the insulators and wire clamps are located in combination with the second area and the second infrared area; and merging the target area of ​​the infrared image with the target area of ​​the visible light image to obtain a target image.

[0088] It should be noted that the computer-readable medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0089] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0090] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0091] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and the one or more programs include instructions, and when the instructions are executed by the electronic device, the electronic device implements the method described in the above embodiment.

[0092] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0093] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An image processing method, characterized in that: include: Obtain infrared images and visible light images of the power transmission lines in the inspection area; Performing target detection on the visible light image to obtain a first area where the power transmission line is located in the visible light image, and performing target detection on the infrared image to obtain a first infrared area where the power transmission line is located in the infrared image; Performing target detection on the first area in the visible light image to obtain a second area where the detection target is located, and performing target detection on the first infrared area in the infrared image to obtain a second infrared area where an insulator and a wire clamp are located; Wherein, the detection targets include insulators and wire clamps; Determine the target area where the insulator and the wire clamp are located by combining the second area and the second infrared area; The target area of ​​the infrared image is fused with the target area of ​​the visible light image to obtain a target image.

2. The image processing method according to claim 1, characterized in that: The performing target detection on the first area in the visible light image to obtain a second area where the insulator and the wire clamp are located, and the performing target detection on the first infrared area in the infrared image to obtain a second infrared area where the insulator and the wire clamp are located, include: Determine a detection area where the power transmission line is located by combining the first area and the first infrared area; Target detection is performed on the detection area in the visible light image to obtain a second area where the insulator and the wire clamp are located, and target detection is performed on the detection area in the infrared image to obtain a second infrared area where the insulator and the wire clamp are located.

3. The image processing method according to claim 1, characterized in that: The performing target detection on the visible light image to obtain a first area where the power transmission line is located in the visible light image includes: Calculating a binarization threshold based on each pixel point of the visible light image, and binarizing the visible light image using the binarization threshold to obtain a first image; A first area where a power transmission line is located is determined from the first image.

4. The image processing method according to claim 1, characterized in that: The performing target detection on the first area in the visible light image to obtain a second area where the detected target is located includes: Calculating a binarization threshold based on pixel points in the first region of the visible light image, and binarizing the first region using the binarization threshold to obtain a second image; A second area where the detection target is located is determined from the second image.

5. The image processing method according to claim 1, characterized in that: Also includes: Using the target image to identify the current use status of the insulator and the clamp; When there is an abnormality in the use state, position information corresponding to the target image is obtained.

6. The image processing method according to claim 1, characterized in that: The step of fusing the target area of ​​the infrared image with the target area of ​​the visible light image to obtain a target image includes: The pixel points in the target area of ​​the visible light image and the pixel points in the target area of ​​the infrared image are weighted averaged to obtain a fused target image.

7. The image processing method according to claim 6, characterized in that: The step of fusing the target area of ​​the infrared image with the target area of ​​the visible light image to obtain a target image includes: The weighted averaged pixels are converted to the HIS color space to obtain a target image.

8. The image processing method according to claim 1, characterized in that: The step of obtaining infrared images and visible light images of power transmission lines in the inspection area includes: Obtain infrared images and visible light images of the power transmission lines in the inspection area; Extracting the maximum and minimum values ​​of pixels in the infrared image; Calculating a stretch ratio by using the maximum value and the minimum value, and updating a pixel value of each pixel in the infrared image based on the stretch ratio to obtain an updated infrared image; The visible light image is filtered to obtain a processed visible light image.

9. An image processing device, characterized in that: include: An image acquisition module is used to acquire infrared images and visible light images of the power transmission lines in the inspection area; A first detection module is used to perform target detection on the visible light image to obtain a first area where the power transmission line is located in the visible light image, and to perform target detection on the infrared image to obtain a first infrared area where the power transmission line is located in the infrared image; A second detection module is used to perform target detection on the first area in the visible light image to obtain a second area where the detection target is located, and to perform target detection on the first infrared area in the infrared image to obtain a second infrared area where the insulator and the wire clamp are located; Wherein, the detection targets include insulators and wire clamps; A target positioning module, used for determining a target area where the insulator and the wire clamp are located in combination with the second area and the second infrared area; The image fusion module is used to fuse the target area of ​​the infrared image with the target area of ​​the visible light image to obtain a target image.

10. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein one or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the electronic device, the electronic device executes the image processing method according to any one of claims 1 to 8.

Images