Fault equipment identification method based on infrared image

Abstract

The invention discloses a fault equipment identification method based on an infrared image, and belongs to the technical field of power inspection, and the method comprises the steps: obtaining an infrared image photographed by the inspection of an unmanned plane, and carrying out the preprocessing of the infrared image, and obtaining an identification image; performing coordinate conversion based on geographic coordinates of equipment corresponding to the identification image to obtain main body pixel points; constructing a pixel coordinate system based on the main body pixel points, and projecting the identification image to the pixel coordinate system based on the shooting parameters to obtain an identification area; determining a main body pixel point set according to an identification criterion and the identification area; analyzing the fault condition of the corresponding equipment according to the temperature data corresponding to the main body pixel point set and the fault criterion to obtain an analysis result; generating a fault report based on an analysis result of the fault condition; according to the method, the device and the background in the infrared image are separated through the pixel point distribution characteristics of the infrared image, the area where the device is located is analyzed to obtain the fault report, and the accuracy of fault analysis is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of electric power inspection, and in particular to a method for identifying faulty equipment based on infrared images. Background Art

[0002] As the scale of power systems continues to expand, the requirements for inspection and maintenance of power equipment are increasing. Using drones equipped with infrared thermal imagers for power inspections has become an important means, but there are currently many deficiencies in infrared image analysis and processing. Especially in complex environments affected by factors such as environmental background, ambient light, camera visual angle, and noise, traditional analysis methods have difficulty accurately distinguishing true fault hotspots, resulting in a high false alarm rate, affecting inspection efficiency and accuracy, and failing to meet the needs of efficient and accurate power operation and maintenance.

[0003] Chinese patent, publication number: CN115015700A, publication date: September 6, 2022, discloses a fault diagnosis method for transmission line insulators, including: obtaining an inspection infrared image of the power grid line; obtaining a target area image of the insulator in the inspection infrared image; performing thermal conversion processing on the target area image of the insulator to generate a thermal map, and the thermal map is used to represent the temperature information of the insulator; based on the temperature information represented in the thermal map, generating prompt information, and the prompt information is used to represent the operating status of the insulator; However, this invention does not take into account the pixel distribution characteristics of the infrared image affected by temperature, resulting in low accuracy of fault analysis. Summary of the Invention

[0004] The purpose of the present invention is to address the problem that the existing technology does not take into account the pixel distribution characteristics of infrared images affected by temperature, resulting in low fault analysis accuracy; a method for fault equipment identification based on infrared images is proposed, the infrared image is preprocessed to obtain an identification image, and a pixel coordinate system is constructed based on the geographic coordinates of the device corresponding to the identification image, the main pixel point set is determined based on the shooting parameters, the pixel coordinate system and the identification criteria, the fault condition of the corresponding device is analyzed according to the fault criteria of the temperature data set corresponding to the main pixel point set, and a fault report is generated based on the analysis results; the present invention separates the device and the background in the infrared image through the pixel distribution characteristics of the infrared image, and analyzes the area where the device is located to obtain a fault report, thereby significantly improving the accuracy of fault analysis.

[0005] In a first aspect, a technical solution provided in an embodiment of the present invention is a method for identifying faulty equipment based on infrared images, comprising the following steps: Obtain infrared images taken by drone inspections and pre-process them to obtain recognition images; Based on the geographic coordinates of the device corresponding to the recognized image, coordinate conversion is performed to obtain the main pixel point; Construct a pixel coordinate system based on the subject pixels, project the recognition image onto the pixel coordinate system based on the shooting parameters to obtain a recognition area; determine the subject pixel set based on the recognition criteria and the recognition area; Analyze the fault condition of the corresponding equipment according to the temperature data corresponding to the main pixel set and the fault criteria to obtain the analysis result; A fault report is generated based on the analysis result of the fault condition.

[0006] In this solution, infrared images taken by drones during power inspections are obtained as raw data, and the infrared images are preprocessed to obtain recognition images with noise removed and heating areas enhanced. At the same time, coordinate conversion is performed based on the geographic coordinates of the device corresponding to the recognition image to obtain the main pixel points. The devices are usually multiple devices gathered in the same place, such as insulators, etc., so the multiple devices gathered together and their connecting bodies, such as cement poles, are taken as a whole. The geographic coordinates of the corresponding devices are actually the geographic coordinates of the whole. The obtained main pixel points represent the approximate position of the whole in the corresponding pixel coordinate system. The recognition image is projected onto the pixel coordinate system based on the shooting parameters of the drone to obtain the recognition area. In essence, the infrared image is described in the pixel coordinate system using pixel points. Combined with the fact that the infrared image has not undergone substantial changes, the main pixel points can be used to determine the specific location of the device corresponding to the infrared image in the pixel coordinate system. Position, and the main pixel point is only a pixel point, and the area range of the device cannot be known, so the main pixel point set is determined according to the recognition criteria and the recognition area, and the main pixel point set represents the area and range of the device corresponding to the infrared image in the pixel coordinate system. At this time, the device and the background in the infrared image have actually been separated, that is, the device is regarded as the foreground and the background is regarded as the background for separation; combined with the temperature data in the infrared image taken by the drone, even if the infrared image is projected into the pixel coordinate system, the temperature data of the pixel point at the corresponding position of the infrared image is still accurate, then the fault condition of the corresponding device is analyzed according to the temperature data corresponding to the main pixel point set and the fault criteria to obtain the analysis result, accurately judge the fault condition of the equipment in the infrared image taken during the drone power inspection, and generate a fault report based on the analysis result of the fault condition, which significantly improves the accuracy of fault analysis.

[0007] Preferably, the specific process of preprocessing the infrared image to obtain the recognition image is: The pixel values of the infrared image are linearly mapped to the normalized range to obtain a normalized image, and the normalized image is decomposed based on a preset wavelet basis to obtain low-frequency sub-bands and high-frequency sub-bands; The coefficients of the high-frequency subband are shrunk based on the threshold function to obtain a shrunk subband, and an inverse discrete wavelet transform is performed based on the shrunk subband and the low-frequency subband to obtain a reconstructed image; Determining a heating interval based on the pixel temperature distribution of the reconstructed image, and setting enhancement parameters based on the resolution of the reconstructed image; The reconstructed image is histogram equalized based on the fever interval and enhancement parameters to obtain the recognition image.

[0008] In this scheme, since the pixel values at different positions of the infrared image are different, in order to avoid quantization errors, the pixel values of the infrared image are mapped to the normalized range according to a certain linear relationship to obtain a normalized image. The linear relationship is to scale the pixel values proportionally while maintaining the trend of pixel value changes. The normalized range can be set to 0 to 1; the normalized image is decomposed based on the wavelet basis function to obtain low-frequency sub-bands and high-frequency sub-bands, so that the noise in the image is visible, and then the threshold function is used to select a suitable threshold to shrink the coefficient of the high-frequency sub-band. When the coefficient is greater than the threshold, a new coefficient is obtained based on the coefficient minus the threshold. When the coefficient is less than the threshold, an attenuation exponent is added to the coefficient so that the coefficient is smoothly reduced to 0. When the coefficient is equal to the threshold, a compromise rate is randomly selected and multiplied by the threshold to obtain a compromise threshold, and a new coefficient is obtained based on the coefficient minus the compromise threshold. The new coefficient, the coefficient smoothly reduced to 0 and the high-frequency sub-band are sorted to obtain a compromised image. Shrinking subband, the attenuation index is set according to the historical shrinking process of the high-frequency subband, and the compromise rate is actually a number between 0 and 1. At this time, an inverse discrete wavelet transform is performed based on the shrinking subband and the low-frequency subband to obtain a reconstructed image, and the noise in the infrared image is successfully removed; since the infrared image will produce a corresponding temperature distribution field according to the influence of the external temperature, the pixel temperature distribution of the reconstructed image is statistically obtained to obtain a corresponding pixel temperature distribution histogram, and the average temperature is used as a benchmark to judge the high-temperature area and the low-temperature area in the reconstructed image, thereby determining the heating area, and setting the enhancement parameters based on the resolution of the reconstructed image, which is essentially to set the enhancement parameters such as the tile grid size and clipping limit based on the resolution of the infrared image to avoid the loss of image details during the enhancement process; finally, the reconstructed image is histogram equalized based on the heating interval and the enhancement parameters to obtain a recognition image, and the brightness channel and saturation of the image are adjusted to make the heating area of the reconstructed image prominent.

[0009] Preferably, the threshold function is specifically: T(x)=sign(x)×(x|-τ)×e -kx ; Where T(x) is the threshold, x is the subband coefficient, sign(x) is the sign function, τ is the threshold parameter, k is the attenuation control rate, σ s is the median of the subband coefficients, σ 2 is the sub-band noise variance.

[0010] Preferably, the specific process of performing coordinate conversion based on the geographic coordinates of the device corresponding to the identified image to obtain the subject pixel point is: Establishing a camera coordinate system with the corresponding drone as the center point, and mapping the corresponding device to the camera coordinate system based on the geographic coordinates of the device corresponding to the recognition image and the geographic coordinates of the drone to obtain the device camera coordinates; Converting the device camera coordinates into device image coordinates based on the shooting focal length of the drone, and converting the device image coordinates into device pixel coordinates based on the drone shooting frame; The device pixel coordinates are sorted to obtain the subject pixel point.

[0011] In this solution, when the drone shoots infrared images, the corresponding device and its background enter the drone's viewfinder in the form of projection, resulting in a certain geometric relationship between the device, background and drone. The infrared image is obtained by capturing the thermal radiation of the object, especially in the summer when the outside temperature is high, the infrared image cannot directly obtain the device area. The camera coordinate system is established with the corresponding drone as the center point. In fact, the camera coordinate system is established with the camera taking the picture on the drone as the center point, and the direction of the camera facing the device is taken as the forward direction, with the camera forward as the z positive semi-axis, the camera upward as the y positive semi-axis, and the camera right as the x positive semi-axis. Then, based on the recognition image correspondence The geographic coordinates of the device and the geographic coordinates of the drone will map the corresponding device to the camera coordinate system according to the geometric relationship to obtain the device camera coordinates, and the position of the device relative to the drone will be obtained. The device camera coordinates will be converted into device image coordinates based on the shooting focal length of the drone. When the drone shoots the image, the theoretical position of the corresponding device projected onto the image will be obtained. However, the theoretical position is only a point of the device in the corresponding area of the image. If you want to obtain the area on the infrared image corresponding to the device, you also need to convert the device image coordinates into device pixel coordinates based on the size of the drone shooting frame, and use the pixel coordinate system corresponding to the device pixel coordinates as an intermediate medium to project the device pixel coordinates onto the infrared image.

[0012] Preferably, the specific process of constructing a pixel coordinate system based on the subject pixel points and projecting the recognition image onto the pixel coordinate system based on the shooting parameters to obtain the recognition area is as follows: Determine the pixel origin based on the device pixel coordinates corresponding to the subject pixel, and establish a pixel coordinate system based on the pixel origin; The pixel size of the image is calculated based on the focal length and digital zoom in the drone shooting parameters to obtain the image pixel point set; The image pixel point set is mapped to a pixel coordinate system to obtain a pixel image, and the main pixel points are mapped to the pixel coordinate system; the main pixel points are marked as identification points, and the identification points and the pixel image are sorted to obtain an identification area.

[0013] In this solution, since the size of the drone shooting frame, that is, the size of the infrared image frame, has been obtained, the theoretical pixel origin is obtained based on the device pixel coordinates corresponding to the main pixel point, and the direction of the pixel origin relative to the main pixel point is used as the positive semi-axis direction to establish a pixel coordinate system. At this time, although the pixel coordinate system has been successfully established and it is known that the main pixel point is related to the device main body in the infrared image, the specific position of the main pixel point in the infrared image is unknown. Therefore, the pixel size of the image is calculated based on the focal length and digital zoom in the drone shooting parameters, and the image pixel point set corresponding to the entire infrared image is actually obtained. The main pixel point is used as a pixel point in the image pixel point set, and the image pixel point set is mapped to the pixel coordinate system to obtain a pixel image. The main pixel point is marked as an identification point. Based on the identification point, the approximate area of the device is determined in the pixel image to obtain an identification area, that is, the center area of the circle formed with the main pixel point as the center and the identification distance as the radius. The identification distance is set according to the specific infrared image, and can eliminate areas in the infrared image that are obviously unrelated to the device.

[0014] Preferably, the specific process of determining the subject pixel set according to the recognition criteria and the recognition area is as follows: Sort the pixels in the recognition area based on the vertical coordinates of the pixel coordinate system corresponding to the recognition area, and extract the vertical pixel distance between the pixel and the next pixel based on the sorting order; Determine the vertical subject pixel point based on the vertical pixel distance and the vertical threshold. If the vertical pixel distance is less than the vertical threshold, the pixel point is marked as the vertical subject pixel point. If the vertical pixel distance is greater than or equal to the vertical threshold, the pixel point is marked as the background pixel point. Sort the pixels in the recognition area based on the horizontal coordinates of the pixel coordinate system corresponding to the recognition area, and extract the horizontal pixel distance between the pixel and the next pixel based on the sorting order; Determine the horizontal subject pixel point based on the horizontal pixel distance and the horizontal threshold. If the horizontal pixel distance is less than the horizontal threshold, the pixel point is marked as the horizontal subject pixel point. If the horizontal pixel distance is greater than or equal to the horizontal threshold, the pixel point is marked as the background pixel point. The pixels marked as longitudinal main pixel points and the pixels marked as transverse main pixel points are sorted to obtain a main pixel point set.

[0015] In this solution, the pixel ordering is actually to order the pixels from small to large according to the values on the coordinate axes corresponding to the pixels, and the pixel distance is actually obtained by subtracting the values on the coordinate axes corresponding to the pixels.

[0016] Preferably, the specific process of analyzing the fault condition of the corresponding device according to the temperature data corresponding to the main pixel set and the fault criterion to obtain the analysis result is: Randomly select a pixel point in the main pixel point set as the center of a circle and draw a circle with a preset pixel radius, extract the highest temperature and the lowest temperature corresponding to the pixel point in the circle, and calculate the circle temperature difference by subtracting the highest temperature from the lowest temperature; Determine whether the circle is a boundary circle based on the circle temperature difference and a first temperature difference threshold; if the circle temperature difference is less than or equal to the first temperature difference threshold, determine that the circle is not a boundary circle, and randomly select a pixel point as the center of the circle to draw a circle; if the circle temperature difference is greater than the first temperature difference threshold, determine that the circle is a boundary circle; Arranging the boundary circle to obtain an abnormal area, and extracting the local maximum temperature based on the abnormal window, comparing the local maximum temperature with a temperature threshold, and determining that the pixel corresponding to the abnormal window is an abnormal pixel if the local maximum temperature is greater than or equal to the temperature threshold; and determining that the pixel corresponding to the abnormal window is a normal pixel if the local maximum temperature is less than the temperature threshold; An abnormal circle is drawn with the pixel with the highest temperature among the abnormal pixels as the center, and the abnormal circle is circumscribed to the boundary circle. The highest temperature and the lowest temperature in the abnormal circle are extracted, and the abnormal temperature difference is obtained by subtracting the highest temperature from the lowest temperature. Determine whether the abnormal circle has a defect based on the abnormal temperature difference and the second temperature difference threshold. If the abnormal temperature difference is greater than or equal to the second temperature difference threshold, determine that the abnormal circle has a defect. If the abnormal temperature difference is less than the second temperature difference threshold, determine that the abnormal circle is normal. The abnormal circles with defects are sorted out to obtain analysis results.

[0017] In this solution, given that the device is a continuous and complete area on the infrared image, the boundary circles will be connected to each other to form an area. The abnormal pixels and abnormal circles in the subsequent process are all within the area formed by the boundary circles. The abnormal circles need to be connected to the nearest boundary circle to cover the entire range of the corresponding abnormal device.

[0018] Preferably, the specific process of generating a fault report based on the analysis result of the fault situation is: Extracting an abnormal circle with defects in the fault analysis result, marking the infrared image corresponding to the abnormal circle as a fault image, and marking the device corresponding to the abnormal circle position in the infrared image as a faulty device; Extracting corresponding operation and maintenance information based on the faulty equipment and the operation and maintenance information database; Organize fault images, faulty equipment, and operation and maintenance information to obtain a fault report.

[0019] In the second aspect, a technical solution also provided in an embodiment of the present invention is an electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the processor implements the steps of a faulty equipment identification method based on infrared images.

[0020] In a third aspect, a technical solution also provided in an embodiment of the present invention is a storage medium, in which computer executable instructions are stored. When the computer executable instructions are loaded and executed by a processor, the steps of a method for identifying faulty equipment based on infrared images are implemented.

[0021] Beneficial effects of the present invention: (1) This application establishes a camera coordinate system with the drone as the center point, maps the device corresponding to the identified image to the camera coordinate system to obtain a certain point in the device area corresponding to the infrared image, namely the main pixel point, and uses the pixel coordinate system established based on the main pixel point as an intermediate medium to map the infrared image and the main pixel point to the pixel coordinate system to obtain the identification area of the device corresponding to the infrared image. Combined with the pixel distribution characteristics of the infrared image, the main pixel point set is determined according to the recognition criteria and the recognition area to achieve the separation of the foreground and background of the infrared image, wherein the device is taken as the foreground and the background is taken as the background, and the area where the device is located is analyzed to obtain a fault report, which significantly improves the accuracy of fault analysis; (2) The present application utilizes electronic devices and storage media to implement the steps of a method for identifying faulty equipment based on infrared images, which can analyze the fault conditions of the inspection equipment in real time during the inspection by the drone without transmitting the data to the background. Moreover, the energy consumption of the electronic devices and storage media is very small compared to the energy consumption of the drone itself, and therefore has no effect on the endurance of the drone. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Other features, objects, and advantages of the present invention will become more apparent upon reading the detailed description of the non-limiting embodiments made with reference to the following drawings. The drawings are for the purpose of illustrating preferred embodiments only and are not to be construed as limiting the present invention. Like reference characters are used throughout the drawings to designate like parts.

[0023] Figure 1 The figure is a flowchart of a method for identifying faulty equipment based on infrared images; Figure 2 Schematic diagram of infrared images of photovoltaic panels taken for drone inspection. DETAILED DESCRIPTION

[0024] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific implementation method described herein is only an optimal embodiment of the present invention, which is only used to explain the present invention and does not limit the scope of protection of the present invention. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0025] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the operations (or steps) as sequential processes, many of the operations (or steps) therein can be performed in parallel, concurrently, or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operations are completed, but can also have additional steps not included in the figures; the process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0026] Example 1: like Figure 1 As shown, this embodiment provides a method for identifying faulty equipment based on infrared images, comprising the following steps: obtaining Figure 2 The infrared image of the photovoltaic panel taken by the drone inspection is shown and pre-processed to obtain the recognition image; In one embodiment, the specific process of preprocessing the infrared image to obtain the recognition image is as follows: The pixel values of the infrared image are linearly mapped to the normalized range to obtain a normalized image, and the normalized image is decomposed based on a preset wavelet basis to obtain low-frequency sub-bands and high-frequency sub-bands; The coefficients of the high-frequency subband are shrunk based on the threshold function to obtain a shrunk subband, and an inverse discrete wavelet transform is performed based on the shrunk subband and the low-frequency subband to obtain a reconstructed image; The threshold function is specifically: T(x)=sign(x)×(x|-τ)×e -kx ; Where T(x) is the threshold, x is the subband coefficient, sign(x) is the sign function, τ is the threshold parameter, k is the attenuation control rate, σ s is the median of the subband coefficients, σ 2 is the sub-band noise variance; Determining a heating interval based on the pixel temperature distribution of the reconstructed image, and setting enhancement parameters based on the resolution of the reconstructed image; The reconstructed image is histogram equalized based on the fever interval and enhancement parameters to obtain the recognition image.

[0027] In this embodiment, since the pixel values at different positions of the infrared image are different, in order to avoid quantization errors, the pixel values of the infrared image are mapped to a normalized range according to a certain linear relationship to obtain a normalized image. The linear relationship is to scale the pixel values proportionally while maintaining the trend of pixel value changes. The normalized range can be set to 0 to 1; the normalized image is decomposed based on the wavelet basis function to obtain low-frequency sub-bands and high-frequency sub-bands, so that the noise in the image is visible, and then a suitable threshold is selected using a threshold function to shrink the coefficient of the high-frequency sub-band. When the coefficient is greater than the threshold, a new coefficient is obtained based on the coefficient minus the threshold. When the coefficient is less than the threshold, an attenuation exponent is added to the coefficient so that the coefficient is smoothly reduced to 0. When the coefficient is equal to the threshold, a compromise rate is randomly selected and multiplied by the threshold to obtain a compromise threshold, and a new coefficient is obtained based on the coefficient minus the compromise threshold. The new coefficient, the coefficient smoothly reduced to 0, and the high-frequency sub-band are sorted to obtain a compromised image. Shrinking subband, the attenuation index is set according to the historical shrinking process of the high-frequency subband, and the compromise rate is actually a number between 0 and 1. At this time, an inverse discrete wavelet transform is performed based on the shrinking subband and the low-frequency subband to obtain a reconstructed image, and the noise in the infrared image is successfully removed; since the infrared image will produce a corresponding temperature distribution field according to the influence of the external temperature, the pixel temperature distribution of the reconstructed image is statistically obtained to obtain a corresponding pixel temperature distribution histogram, and the average temperature is used as a benchmark to judge the high-temperature area and the low-temperature area in the reconstructed image, thereby determining the heating area, and setting the enhancement parameters based on the resolution of the reconstructed image, which is essentially to set the enhancement parameters such as the tile grid size and clipping limit based on the resolution of the infrared image to avoid the loss of image details during the enhancement process; finally, the reconstructed image is histogram equalized based on the heating interval and the enhancement parameters to obtain a recognition image, and the brightness channel and saturation of the image are adjusted to make the heating area of the reconstructed image prominent.

[0028] Based on the geographic coordinates of the device corresponding to the recognized image, coordinate conversion is performed to obtain the main pixel point; Specifically, a camera coordinate system is established with the corresponding drone as the center point, and the corresponding device is mapped to the camera coordinate system based on the geographic coordinates of the device corresponding to the recognition image and the geographic coordinates of the drone to obtain the device camera coordinates; Converting the device camera coordinates into device image coordinates based on the shooting focal length of the drone, and converting the device image coordinates into device pixel coordinates based on the drone shooting frame; The device pixel coordinates are sorted to obtain the subject pixel point.

[0029] In this embodiment, when the drone shoots infrared images, the corresponding device and its background enter the drone's viewfinder in the form of projection, resulting in a certain geometric relationship between the device, background and drone. Combined with the infrared image obtained by capturing the thermal radiation of the object, especially in the summer when the outside temperature is high, the infrared image cannot directly obtain the device area, so a camera coordinate system is established with the corresponding drone as the center point. In essence, the camera coordinate system is established with the camera on the drone taking the picture as the center point, and the direction of the camera facing the device is used as the forward direction, with the camera forward as the positive z axis, the camera upward as the positive y axis, and the camera right as the positive x axis. Then, based on the geographic coordinates of the device corresponding to the identified image and the geographic coordinates of the drone, the corresponding device is mapped to the camera coordinate system according to the geometric relationship to obtain the device camera coordinates, and the position of the device relative to the drone is obtained. The device camera coordinates are converted into device image coordinates based on the shooting focal length of the drone to obtain the theoretical position of the corresponding device projected onto the image when the drone shoots the image. The image coordinate calculation formula corresponding to the device image coordinates is specifically as follows: Where x is the horizontal coordinate of the device image coordinate, y is the vertical coordinate of the device image coordinate, f is the focal length of the drone, and x is the vertical coordinate of the device image coordinate. c The horizontal coordinate of the device camera coordinate, y c The vertical coordinate of the device camera coordinate, z c is the vertical coordinate of the device camera coordinates; however, the theoretical position is only a point of the device in the corresponding area of the image. If you want to obtain the area on the infrared image corresponding to the device, you need to convert the device image coordinates into device pixel coordinates based on the size of the drone shooting frame, and use the pixel coordinate system corresponding to the device pixel coordinates as an intermediate medium to project the device pixel coordinates onto the infrared image. The specific calculation formula for the pixel coordinates corresponding to the device pixel coordinates is: Where u is the horizontal coordinate of the device pixel coordinate, v is the vertical coordinate of the device pixel coordinate, imageWidth is the width of the drone shooting frame, imageHeith is the height of the drone shooting frame, and pixelSize indicates how many pixels per mm are equal; the drone shooting frame is usually a known parameter.

[0030] Construct a pixel coordinate system based on the subject's pixels, and project the recognition image onto the pixel coordinate system based on the shooting parameters to obtain the recognition area; Specifically, the pixel origin is determined based on the device pixel coordinates corresponding to the subject pixel point, and a pixel coordinate system is established based on the pixel origin; The pixel size of the image is calculated based on the focal length and digital zoom in the drone shooting parameters to obtain the image pixel point set; The image pixel point set is mapped to a pixel coordinate system to obtain a pixel image, and the main pixel points are mapped to the pixel coordinate system; the main pixel points are marked as identification points, and the identification points and the pixel image are sorted to obtain an identification area.

[0031] In this embodiment, since the size of the drone shooting frame, that is, the size of the infrared image frame, has been obtained, the theoretical pixel origin is obtained based on the device pixel coordinates corresponding to the main pixel point, and the direction of the pixel origin relative to the main pixel point is used as the positive semi-axis direction to establish a pixel coordinate system. At this time, although the pixel coordinate system has been successfully established and it is known that the main pixel point is related to the device main body in the infrared image, the specific position of the main pixel point in the infrared image is unknown. Therefore, the pixel size of the image is calculated based on the focal length and digital zoom in the drone shooting parameters, and the image pixel point set corresponding to the entire infrared image is actually obtained. The calculation formula of the pixel size is specifically as follows: pointSize = pixelSize × J j ×S z ×X; Where pointSize represents the size of the infrared image on the canvas, i.e., the pixel size. The canvas corresponds to the pixel coordinate system. pixelSize represents how many pixels per mm. J j is the focal length, S z is digital zoom, X is the density influence weight, and here the density influence weight is set to 0.011 based on experience; The subject pixel point is taken as a pixel point in the image pixel point set, the image pixel point set is mapped to the pixel coordinate system to obtain a pixel image, the subject pixel point is marked as an identification point, and the approximate area of the device is determined in the pixel image based on the identification point to obtain an identification area, that is, a circle center area formed with the subject pixel point as the center and the identification distance as the radius. The identification distance is set according to the specific infrared image, and areas in the infrared image that are obviously not related to the device can be eliminated.

[0032] Determine the subject pixel point set according to the recognition criteria and the recognition area; Specifically, the pixels in the recognition area are sorted based on the vertical coordinates of the pixel coordinate system corresponding to the recognition area, and the vertical pixel distance between the pixel and the next pixel is extracted based on the sorting order; Determine the vertical subject pixel point based on the vertical pixel distance and the vertical threshold. If the vertical pixel distance is less than the vertical threshold, the pixel point is marked as the vertical subject pixel point. If the vertical pixel distance is greater than or equal to the vertical threshold, the pixel point is marked as the background pixel point. Sort the pixels in the recognition area based on the horizontal coordinates of the pixel coordinate system corresponding to the recognition area, and extract the horizontal pixel distance between the pixel and the next pixel based on the sorting order; Determine the horizontal subject pixel point based on the horizontal pixel distance and the horizontal threshold. If the horizontal pixel distance is less than the horizontal threshold, the pixel point is marked as the horizontal subject pixel point. If the horizontal pixel distance is greater than or equal to the horizontal threshold, the pixel point is marked as the background pixel point. The pixels marked as longitudinal main pixel points and the pixels marked as transverse main pixel points are sorted to obtain a main pixel point set.

[0033] In this embodiment, the pixel sorting is essentially sorting the pixels from small to large according to the values on the coordinate axes corresponding to the pixels. The pixel distance is actually obtained by subtracting the values on the coordinate axes corresponding to the pixels. The longitudinal threshold and the transverse threshold are both set to M pixels and calculated based on the pixel size. The calculation formula for M pixels is specifically: M = pointSize × 2.5; Where pointSize is the pixel size.

[0034] Analyze the fault condition of the corresponding device based on the temperature data corresponding to the main pixel set and the fault criteria to obtain an analysis result; specifically, randomly select a pixel point in the main pixel set as the center of a circle with a preset pixel radius, extract the maximum temperature and minimum temperature corresponding to the pixel points in the circle, and subtract the maximum temperature from the minimum temperature to obtain the circle temperature difference; Determine whether the circle is a boundary circle based on the circle temperature difference and a first temperature difference threshold; if the circle temperature difference is less than or equal to the first temperature difference threshold, determine that the circle is not a boundary circle, and randomly select a pixel point as the center of the circle to draw a circle; if the circle temperature difference is greater than the first temperature difference threshold, determine that the circle is a boundary circle; Arranging the boundary circle to obtain an abnormal area, and extracting the local maximum temperature based on the abnormal window, comparing the local maximum temperature with a temperature threshold, and determining that the pixel corresponding to the abnormal window is an abnormal pixel if the local maximum temperature is greater than or equal to the temperature threshold; and determining that the pixel corresponding to the abnormal window is a normal pixel if the local maximum temperature is less than the temperature threshold; An abnormal circle is drawn with the pixel with the highest temperature among the abnormal pixels as the center, and the abnormal circle is circumscribed to the boundary circle. The highest temperature and the lowest temperature in the abnormal circle are extracted, and the abnormal temperature difference is obtained by subtracting the highest temperature from the lowest temperature. Determine whether the abnormal circle has a defect based on the abnormal temperature difference and the second temperature difference threshold. If the abnormal temperature difference is greater than or equal to the second temperature difference threshold, determine that the abnormal circle has a defect. If the abnormal temperature difference is less than the second temperature difference threshold, determine that the abnormal circle is normal. The abnormal circles with defects are sorted out to obtain analysis results.

[0035] In this embodiment, a pixel point in the main pixel point set is randomly selected as the center of a circle, and a circle is drawn with a radius of 30px pixels until all the pixel points in the main pixel point set are used as the center of the circle. It is judged whether the temperature difference between the highest temperature and the lowest temperature in each circle is less than 20°C. If so, the range of the pixel points is expanded outward based on the circle. At this time, some pixel points that belong to the infrared image but do not belong to the main pixel point set are included in the main pixel point set to obtain a main pixel point set with a larger range. The process of drawing circles and judging is repeated until all circles with a temperature difference between the highest temperature and the lowest temperature greater than or equal to 20°C are found, that is, boundary circles. The boundary circles are connected to each other and can form a circle with the boundary circle as the boundary. A complete and closed area. When the complete and closed area appears, it proves that all the circumscribed circles have been found. The complete and closed area is actually the abnormal area to be detected; the temperature threshold can be set to 80℃, and all pixels with temperatures greater than or equal to 80℃ are found. These pixel points correspond to the positions of abnormal equipment, and an abnormal circle is drawn with the pixel point as the center. The radius of the abnormal circle is preferably such that it is connected to the nearest circumscribed circle, so that the faulty equipment can be covered within the range of the abnormal circle. Then, based on the temperature difference between the highest temperature and the lowest temperature in the abnormal circle and the second temperature difference threshold also set to 20℃, the corresponding area is checked to see if it is abnormal, and the abnormal circle that is verified to be abnormal is marked as defective.

[0036] generating a fault report based on the analysis result of the fault condition; Specifically, extracting an abnormal circle with defects in the fault analysis result, marking the infrared image corresponding to the abnormal circle as a fault image, and marking the device corresponding to the abnormal circle position in the infrared image as a faulty device; Extracting corresponding operation and maintenance information based on the faulty equipment and the operation and maintenance information database; Organize fault images, faulty equipment, and operation and maintenance information to obtain a fault report.

[0037] In the second aspect, a technical solution also provided in an embodiment of the present invention is an electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the processor implements the steps of a faulty equipment identification method based on infrared images.

[0038] In a third aspect, a technical solution also provided in an embodiment of the present invention is a storage medium, in which computer executable instructions are stored. When the computer executable instructions are loaded and executed by a processor, the steps of a method for identifying faulty equipment based on infrared images are implemented.

[0039] This embodiment has at least the following substantial effects: (1) This embodiment establishes a camera coordinate system with the drone as the center point, maps the device corresponding to the identified image to the camera coordinate system to obtain a certain point in the device area corresponding to the infrared image, namely the main pixel point, and uses the pixel coordinate system established based on the main pixel point as an intermediate medium to map the infrared image and the main pixel point to the pixel coordinate system to obtain the identification area of the device corresponding to the infrared image. Combined with the pixel distribution characteristics of the infrared image, the main pixel point set is determined according to the recognition criteria and the recognition area to achieve the separation of the foreground and background of the infrared image, wherein the device is taken as the foreground and the background is taken as the background, and the area where the device is located is analyzed to obtain a fault report, which significantly improves the accuracy of fault analysis; (2) This embodiment utilizes electronic devices and storage media to implement the steps of a method for identifying faulty equipment based on infrared images. The fault conditions of the inspection equipment can be analyzed in real time during the inspection by the drone without transmitting the data to the background. Moreover, the energy consumption of the electronic devices and storage media is very small compared to the energy consumption of the drone itself, so it has no effect on the endurance of the drone.

[0040] The above specific embodiments are preferred embodiments of the present invention and are not intended to limit the specific implementation scope of the present invention. The scope of the present invention includes but is not limited to the specific embodiments. All equivalent changes made in accordance with the shape, structure, and method of the present invention are within the scope of protection of the present invention.

Claims

1. A method for identifying faulty equipment based on infrared images, characterized in that: The following steps are involved: Obtain infrared images taken by drone inspections and pre-process them to obtain recognition images; Based on the geographic coordinates of the device corresponding to the recognized image, coordinate conversion is performed to obtain the main pixel point; Construct a pixel coordinate system based on the subject pixels, project the recognition image onto the pixel coordinate system based on the shooting parameters to obtain a recognition area; determine the subject pixel set based on the recognition criteria and the recognition area; Analyze the fault condition of the corresponding equipment according to the temperature data corresponding to the main pixel set and the fault criteria to obtain the analysis result; A fault report is generated based on the analysis result of the fault condition.

2. The method for identifying faulty equipment based on infrared images according to claim 1, characterized in that: The specific process of preprocessing the infrared image to obtain the recognition image is as follows: The pixel values of the infrared image are linearly mapped to the normalized range to obtain a normalized image, and the normalized image is decomposed based on a preset wavelet basis to obtain low-frequency sub-bands and high-frequency sub-bands; The coefficients of the high-frequency subband are shrunk based on the threshold function to obtain a shrunk subband, and an inverse discrete wavelet transform is performed based on the shrunk subband and the low-frequency subband to obtain a reconstructed image; Determining a heating interval based on the pixel temperature distribution of the reconstructed image, and setting enhancement parameters based on the resolution of the reconstructed image; The reconstructed image is histogram equalized based on the fever interval and enhancement parameters to obtain the recognition image.

3. The method for identifying faulty equipment based on infrared images according to claim 2, characterized in that: The threshold function is specifically: T(x)=sign(x)×(x-τ)×e -kx ; Where T(x) is the threshold, x is the subband coefficient, sign(x) is the sign function, τ is the threshold parameter, k is the attenuation control rate, σ s is the median of the subband coefficients, σ 2 is the sub-band noise variance.

4. The method for identifying faulty equipment based on infrared images according to claim 1, characterized in that: The specific process of performing coordinate conversion based on the geographic coordinates of the device corresponding to the identified image to obtain the main pixel point is: Establishing a camera coordinate system with the corresponding drone as the center point, and mapping the corresponding device to the camera coordinate system based on the geographic coordinates of the device corresponding to the recognition image and the geographic coordinates of the drone to obtain the device camera coordinates; Converting the device camera coordinates into device image coordinates based on the shooting focal length of the drone, and converting the device image coordinates into device pixel coordinates based on the drone shooting frame; The device pixel coordinates are sorted to obtain the subject pixel point.

5. The method for identifying faulty equipment based on infrared images according to claim 1, characterized in that: The specific process of constructing a pixel coordinate system based on the subject pixel points and projecting the recognition image onto the pixel coordinate system based on the shooting parameters to obtain the recognition area is as follows: Determine the pixel origin based on the device pixel coordinates corresponding to the subject pixel, and establish a pixel coordinate system based on the pixel origin; The pixel size of the image is calculated based on the focal length and digital zoom in the drone shooting parameters to obtain the image pixel point set; Map the image pixel point set to the pixel coordinate system to obtain a pixel image, and map the subject pixel points to the pixel coordinate system; The subject pixel points are marked as identification points, and the identification points and pixel images are sorted to obtain an identification area.

6. The method for identifying faulty equipment based on infrared images according to claim 1, characterized in that: The specific process of determining the subject pixel set according to the recognition criteria and the recognition area is as follows: Sort the pixels in the recognition area based on the vertical coordinates of the pixel coordinate system corresponding to the recognition area, and extract the vertical pixel distance between the pixel and the next pixel based on the sorting order; Determine the vertical subject pixel point based on the vertical pixel distance and the vertical threshold. If the vertical pixel distance is less than the vertical threshold, the pixel point is marked as the vertical subject pixel point. If the vertical pixel distance is greater than or equal to the vertical threshold, the pixel point is marked as the background pixel point. Sort the pixels in the recognition area based on the horizontal coordinates of the pixel coordinate system corresponding to the recognition area, and extract the horizontal pixel distance between the pixel and the next pixel based on the sorting order; Determine the horizontal subject pixel point based on the horizontal pixel distance and the horizontal threshold. If the horizontal pixel distance is less than the horizontal threshold, the pixel point is marked as the horizontal subject pixel point. If the horizontal pixel distance is greater than or equal to the horizontal threshold, the pixel point is marked as the background pixel point. The pixels marked as longitudinal main pixel points and the pixels marked as transverse main pixel points are sorted to obtain a main pixel point set.

7. The method for identifying faulty equipment based on infrared images according to claim 1, characterized in that: The specific process of analyzing the fault condition of the corresponding device according to the temperature data corresponding to the main pixel point set and the fault criterion to obtain the analysis result is as follows: randomly selecting a pixel point in the main pixel point set as the center of a circle with a preset pixel radius, extracting the maximum temperature and the minimum temperature corresponding to the pixel point in the circle, and subtracting the maximum temperature from the minimum temperature to obtain the circle temperature difference; Determine whether the circle is a boundary circle based on the circle temperature difference and a first temperature difference threshold; if the circle temperature difference is less than or equal to the first temperature difference threshold, determine that the circle is not a boundary circle, and randomly select a pixel point as the center of the circle to draw a circle; if the circle temperature difference is greater than the first temperature difference threshold, determine that the circle is a boundary circle; Arranging the boundary circle to obtain an abnormal area, and extracting the local maximum temperature based on the abnormal window, comparing the local maximum temperature with a temperature threshold, and determining that the pixel corresponding to the abnormal window is an abnormal pixel if the local maximum temperature is greater than or equal to the temperature threshold; and determining that the pixel corresponding to the abnormal window is a normal pixel if the local maximum temperature is less than the temperature threshold; An abnormal circle is drawn with the pixel with the highest temperature among the abnormal pixels as the center, and the abnormal circle is circumscribed to the boundary circle. The highest temperature and the lowest temperature in the abnormal circle are extracted, and the abnormal temperature difference is obtained by subtracting the highest temperature from the lowest temperature. Determine whether the abnormal circle has a defect based on the abnormal temperature difference and the second temperature difference threshold. If the abnormal temperature difference is greater than or equal to the second temperature difference threshold, determine that the abnormal circle has a defect. If the abnormal temperature difference is less than the second temperature difference threshold, determine that the abnormal circle is normal. The abnormal circles with defects are sorted out to obtain analysis results.

8. The method for identifying faulty equipment based on infrared images according to claim 1, characterized in that: The specific process of generating a fault report based on the analysis result of the fault situation is as follows: Extracting an abnormal circle with defects in the fault analysis result, marking the infrared image corresponding to the abnormal circle as a fault image, and marking the device corresponding to the abnormal circle position in the infrared image as a faulty device; Extracting corresponding operation and maintenance information based on the faulty equipment and the operation and maintenance information database; Organize fault images, faulty equipment, and operation and maintenance information to obtain a fault report.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the method implements the steps of a faulty equipment identification method based on infrared images as described in any one of claims 1 to 8.

10. A storage medium, characterized in that: The storage medium stores computer-executable instructions, which, when loaded and executed by the processor, implement the steps of a faulty equipment identification method based on infrared images as claimed in any one of claims 1 to 8.

Images