Device Fault Location Method and Device, Storage Medium, and Terminal

Abstract

The present invention discloses a method and device for equipment fault location, a storage medium, and a terminal, which relate to the technical field of power equipment fault detection. The main purpose is to solve the problem of relatively low accuracy in equipment fault location. It mainly includes responding to the fault warning information of the target equipment, obtaining the first-perspective infrared image of the target equipment and the three-dimensional image of the target equipment; determining the first temperature distribution image of the target equipment by performing image registration on the first-perspective infrared image and the three-dimensional image of the equipment, and determining the fault category of the target equipment according to the first temperature distribution image; if the fault category is an internal equipment fault, obtaining the second-perspective infrared image of the target equipment based on the first temperature distribution image, and determining the fault location information of the target equipment according to the first temperature distribution image and the feature matching result between the second-perspective infrared image and the three-dimensional image of the equipment. It is mainly used for the location of power equipment faults.

Description

Technical Field

[0001] The present invention relates to the technical field of power equipment fault detection, and particularly to a method and device for locating equipment faults, a storage medium, and a terminal. Background Art

[0002] Since the faults of power equipment usually manifest as abnormal equipment temperatures, and infrared images can directly reflect the distribution of the radiation energy (temperature) of power equipment, the infrared thermal imaging technology of equipment is applied to the fault detection of power equipment. Due to the advantages of non-contact detection, no electromagnetic interference, large detection range, all-weather operation, intuitive and reliable, etc., the infrared thermal imaging technology of equipment has now become an important means for the fault detection of power equipment.

[0003] Existing power equipment fault detection methods based on infrared thermal imaging technology need to first make a preliminary judgment on the fault location through infrared images, and then the staff further judge the specific location where the fault occurs and the equipment where the fault occurs according to on-site experience. However, due to the uneven experience levels of the staff, the accuracy of fault location is relatively low. Summary of the Invention

[0004] In view of this, the present invention provides a method and device for locating equipment faults, a storage medium, and a terminal, mainly aiming to solve the existing problems.

[0005] According to one aspect of the present invention, a method for locating equipment faults is provided, including:

[0006] In response to the fault warning information of the target equipment, obtain the first perspective infrared image of the target equipment and the three-dimensional image of the target equipment;

[0007] Determine the first temperature distribution image of the target equipment by performing image registration on the first perspective infrared image and the three-dimensional image of the equipment, and determine the fault category of the target equipment according to the first temperature distribution image;

[0008] If the fault category is an internal equipment fault, obtain the second perspective infrared image of the target equipment based on the first temperature distribution image, and determine the fault location information of the target equipment according to the first temperature distribution image and the feature matching result between the second perspective infrared image and the three-dimensional image of the equipment.

[0009] Further, the step of determining the first temperature distribution image of the target equipment by performing image registration on the first perspective infrared image and the three-dimensional image of the equipment includes:

[0010] Extract multiple first structural features of the target device from the first - perspective infrared image by using an image feature extraction model, and extract multiple second structural features of the target device from the three - dimensional image of the device;

[0011] Calculate the pixel matching distance between the pixel area of the first structural feature and the pixel area of the second structural feature;

[0012] If the pixel matching distance is less than or equal to a preset pixel matching threshold, determine the plane where the second structural feature is located as the target plane;

[0013] Perform image registration processing on the first - perspective infrared image and the view of the target device mapped on the target plane to obtain a first temperature distribution image.

[0014] Further, determining the fault category of the target device according to the first temperature distribution image includes:

[0015] Determine the area with a temperature greater than the preset abnormal temperature threshold in the first temperature distribution image as the abnormal temperature pixel area, and extract the edge irregularity and pixel proportion of the abnormal temperature pixel area. The pixel proportion is the pixel proportion of the abnormal temperature pixel area in the first temperature distribution image;

[0016] If the edge irregularity is less than or equal to a preset edge irregularity threshold and the pixel proportion is greater than or equal to a preset pixel proportion, determine the fault category as an internal device fault;

[0017] If the edge irregularity is greater than a preset edge irregularity threshold and the pixel proportion is less than a preset pixel proportion, determine the fault category as an external device fault.

[0018] Further, if the fault category is an internal device fault, obtaining a second - perspective infrared image of the target device based on the first temperature distribution image includes:

[0019] Obtain the first - perspective information of the first temperature distribution image, and calculate the second - perspective information according to the first - perspective information. The second - perspective information represents the perspective perpendicular to the plane corresponding to the first - perspective information;

[0020] Obtain a second - perspective infrared image by sending a second - perspective infrared image acquisition request to the control terminal of the infrared image acquisition device. The second - perspective infrared image acquisition request is generated based on the second - perspective information.

[0021] Further, determining the fault location information of the target device according to the feature matching results of the first temperature distribution image, the second - perspective infrared image and the three - dimensional image of the device includes:

[0022] Perform image registration based on the three-dimensional image of the device and the second-view infrared image to determine the second temperature distribution image of the target device;

[0023] Construct a three-dimensional temperature distribution image of the target device based on the first temperature distribution image and the second temperature distribution image;

[0024] Perform inversion calculation according to the three-dimensional temperature distribution image and the medium temperature propagation parameters to determine the fault location information of the target device.

[0025] Further, when the target device is an electrical device, before obtaining the first-view infrared image of the target device and the three-dimensional image of the target device in response to the fault warning information of the target device, the method further includes:

[0026] Based on the weight parameters and load parameters corresponding to each electrical device, determine the sampling frame number and sampling time for the infrared image acquisition device to perform image acquisition on each electrical device;

[0027] Generate a sampling control instruction based on the sampling frame number and the sampling time, and send the sampling control instruction to the infrared image acquisition device to control the infrared image acquisition device to acquire the first-view infrared image of the corresponding electrical device according to the sampling frame number and the sampling time;

[0028] If the maximum value of the temperature data in the first-view infrared image is greater than or equal to the preset fault temperature threshold, determine the electrical device corresponding to the first-view infrared image as the target device and generate a target device warning information.

[0029] Further, after determining the fault location information of the target device according to the feature matching result between the first temperature distribution image, the second-view infrared image and the three-dimensional image of the device, the method further includes:

[0030] Extract the abnormal temperature, spatial location information corresponding to the fault location, and the component structure image of the device component where the fault location is located;

[0031] Based on the comparison result between the abnormal temperature and multiple preset temperature thresholds, determine the fault state, and the preset temperature thresholds correspond to the fault states representing different fault degrees;

[0032] Use an image semantic segmentation model to perform semantic recognition on the component structure image to obtain the component semantic information of the faulty device component;

[0033] Generate device maintenance information based on the component semantic information and the spatial location information, and output the device maintenance information to a display terminal so that maintenance personnel can repair or replace the device components.

[0034] According to another aspect of the present invention, there is provided a device fault location device, including:

[0035] An acquisition module, configured to acquire a first perspective infrared image of the target device and a three-dimensional device image of the target device in response to a fault warning message of the target device;

[0036] A first determination module, configured to determine a first temperature distribution image of the target device by performing image registration on the first perspective infrared image and the three-dimensional device image, and determine the fault category of the target device according to the first temperature distribution image;

[0037] A second determination module, configured to, if the fault category is an internal device fault, acquire a second perspective infrared image of the target device based on the first temperature distribution image, and determine the fault location information of the target device according to the first temperature distribution image and the feature matching result between the second perspective infrared image and the three-dimensional device image.

[0038] Further, the first determination module includes:

[0039] A first extraction unit, configured to extract a plurality of first structural features of the target device in the first perspective infrared image and a plurality of second structural features of the target device in the three-dimensional device image by using an image feature extraction model;

[0040] A calculation unit, configured to calculate the pixel matching distance between the pixel region of the first structural feature and the pixel region of the second structural feature;

[0041] A first determination unit, configured to, if the pixel matching distance is less than or equal to a preset pixel matching threshold, determine the plane where the second structural feature is located as the target plane;

[0042] A registration unit, configured to perform image registration processing on the first perspective infrared image and the view of the target device mapped on the target plane to obtain a first temperature distribution image.

[0043] Further, the first determination module further includes:

[0044] A second extraction unit, configured to determine, in the first temperature distribution image, an area where the temperature is greater than a preset abnormal temperature threshold as an abnormal temperature pixel area, and extract the edge irregularity and pixel ratio of the abnormal temperature pixel area, where the pixel ratio is the pixel ratio of the abnormal temperature pixel area in the first temperature distribution image;

[0045] A second determination unit, configured to determine that the fault category is an internal fault of the device if the edge irregularity is less than or equal to a preset edge irregularity threshold and the pixel ratio is greater than or equal to a preset pixel ratio;

[0046] A third determination unit, configured to determine that the fault category is an external fault of the device if the edge irregularity is greater than a preset edge irregularity threshold and the pixel ratio is less than a preset pixel ratio.

[0047] Further, the second determination module includes:

[0048] A first acquisition unit, configured to acquire first perspective information of the first temperature distribution image, and calculate second perspective information according to the first perspective information, where the second perspective information represents a perspective perpendicular to a plane corresponding to the first perspective information;

[0049] A second acquisition unit, configured to acquire a second perspective infrared image by sending a second perspective infrared image acquisition request to a control terminal of an infrared image acquisition device, where the second perspective infrared image acquisition request is generated based on the second perspective information.

[0050] Further, the second determination module includes:

[0051] A fourth determination unit, configured to perform image registration according to the three-dimensional image of the device and the second perspective infrared image to determine a second temperature distribution image of the target device;

[0052] A construction unit, configured to construct a three-dimensional temperature distribution image of the target device based on the first temperature distribution image and the second temperature distribution image;

[0053] A fifth determination unit, configured to perform inversion calculation according to the three-dimensional temperature distribution image and a medium temperature propagation parameter to determine fault location information of the target device.

[0054] Further, the device further includes:

[0055] The determination module is further configured to determine the sampling number of frames and sampling time for the infrared image acquisition device to perform image acquisition on each power device based on the weight parameter and load parameter corresponding to each power device;

[0056] A generation module, configured to generate a sampling control instruction based on the number of sampled frames and the sampling time, and send the sampling control instruction to the infrared image acquisition device to control the infrared image acquisition device to acquire first - perspective infrared images of corresponding power equipment according to the number of sampled frames and the sampling time;

[0057] An early - warning module, configured to determine that the power equipment corresponding to the first - perspective infrared image is a target device and generate target - device early - warning information if the maximum value of the temperature data in the first - perspective infrared image is greater than or equal to a preset fault temperature threshold.

[0058] Further, the device further includes:

[0059] An extraction module, configured to extract abnormal temperature, spatial - position information corresponding to the fault location, and a component - structure image of the device component where the fault location is located;

[0060] The determination module is further configured to determine a fault state based on a comparison result between the abnormal temperature and multiple preset temperature thresholds, where the preset temperature thresholds correspond to fault states representing different degrees of faults;

[0061] A semantic - segmentation module, configured to perform semantic recognition on the component - structure image by using an image semantic - segmentation model to obtain component - semantic information of the faulty device component;

[0062] The generation module is further configured to generate device - repair information based on the component - semantic information and the spatial - position information, and output the device - repair information to a display terminal so that maintenance personnel can repair or replace the device component.

[0063] According to another aspect of the present invention, there is provided a storage medium in which at least one executable instruction is stored, and the executable instruction causes a processor to perform operations corresponding to the above - mentioned device - fault location method.

[0064] According to still another aspect of the present invention, there is provided a terminal, including: a processor, a memory, a communication interface, and a communication bus, where the processor, the memory, and the communication interface complete communication with each other through the communication bus;

[0065] The memory is used to store at least one executable instruction, and the executable instruction causes the processor to perform operations corresponding to the above - mentioned device - fault location method.

[0066] By means of the above - mentioned technical solutions, the technical solutions provided by the embodiments of the present invention have at least the following advantages:

[0067] The present invention provides a method and device for equipment fault location, a storage medium, and a terminal. In the embodiments of the present invention, in response to the fault warning information of a target device, a first-perspective infrared image of the target device and a three-dimensional image of the device are acquired; by performing image registration on the first-perspective infrared image and the three-dimensional image of the device, a first temperature distribution image of the target device is determined, and the fault category of the target device is determined according to the first temperature distribution image; if the fault category is an internal equipment fault, a second-perspective infrared image of the target device is acquired based on the first temperature distribution image, and the fault location information of the target device is determined according to the first temperature distribution image and the feature matching result between the second-perspective infrared image and the three-dimensional image of the device, which can greatly reduce the dependence on manual work in equipment fault location, avoid the inaccuracy of manual experience judgment, and achieve precise location of the fault position, thereby reducing the labor cost while effectively improving the accuracy of power equipment fault location.

[0068] The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features, and advantages of the present invention more obvious and understandable, the following specifically illustrates the embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] By reading the detailed description of the preferred embodiments below, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention. And throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0070] Figure 1 shows a flowchart of a method for equipment fault location provided by an embodiment of the present invention;

[0071] Figure 2 shows a flowchart of another method for equipment fault location provided by an embodiment of the present invention;

[0072] Figure 3 shows a flowchart of yet another method for equipment fault location provided by an embodiment of the present invention;

[0073] Figure 4 shows a block diagram of a device for equipment fault location provided by an embodiment of the present invention;

[0074] Figure 5 shows a schematic structural diagram of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0075] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be fully conveyed to those skilled in the art.

[0076] For the existing power equipment fault detection method based on infrared thermal imaging technology, it is necessary to first make a preliminary judgment on the fault location through infrared images, and then the staff further judges the specific location where the fault occurs and the equipment where the fault occurs according to on-site experience. However, due to the uneven experience levels of the staff, the accuracy of fault location is relatively low. Embodiments of the present invention provide a device fault location method, as Figure 1 shown, the method includes:

[0077] 101. In response to the fault warning information of the target device, obtain the first-view infrared image of the target device and the three-dimensional image of the target device.

[0078] In the embodiments of the present invention, when receiving the fault warning information, in order to determine the fault location, the first-view infrared image carried in the fault warning information and the information of the target device are obtained, and according to the information of the target device, the three-dimensional image of the target device is obtained from the pre-constructed device three-dimensional model database. Among them, the acquisition angle of the first-view infrared image is the angle when the fault of the target device is found. Among them, the infrared image is a key-frame infrared image extracted by intermittently extracting frames from the infrared video of the device. The key-frame image can be the image with the largest abnormal temperature value. The embodiments of the present invention do not make specific limitations.

[0079] It should be noted that the device three-dimensional model database is constructed by using three-dimensional model software such as CAD models and GIM models, and includes the device structures of all power equipment and the three-dimensional model images of the spatial connection relationships between device components. The device three-dimensional model images in the database support scaling and viewing at different precisions. Since the infrared image is a planar image and cannot reflect the internal structure of the device, by pre-constructing the device three-dimensional model database, the three-dimensional structure of the device can be provided for determining the internal fault location of the device, so as to achieve accurate identification of the internal fault location of the device.

[0080] 102. Determine the first temperature distribution image of the target device by performing image registration on the first-view infrared image and the three-dimensional image of the device, and determine the fault category of the target device according to the first temperature distribution image.

[0081] In the embodiments of the present invention, due to problems such as few gray imaging pixels, small viewing angle, lack of texture information, and low contrast in infrared images, the image information of the device cannot be accurately displayed. Therefore, through image registration, the first-view infrared image is feature-fused with the three-dimensional image of the device, so as to obtain a planar image containing the device structure information, spatial connection relationship, and temperature distribution information under the current viewing angle, that is, the first temperature distribution image. Before registering with the three-dimensional image of the device, preprocessing such as adjusting the contrast and brightness of the first-view infrared image and upsampling and downsampling can also be performed to enhance the image effect, which is not specifically limited in the embodiments of the present invention. After obtaining the first temperature distribution image, in order to determine the exact location where the fault occurs, the fault type needs to be determined first. Since the device with a fault may be an external device with a large degree of exposure, such as device components like wire clamps and terminal connectors, or it may be a large device, such as the internal device components of a transformer. And the methods for determining the fault location of external devices and internal devices are different. Therefore, it is necessary to determine the fault type, that is, internal fault or external fault, so as to identify the fault location in different ways for different fault types.

[0082] It should be noted that by performing image registration on the first-view infrared image and the three-dimensional image of the device, the problem of poor clarity of the infrared image can be compensated, an accurate determination of the fault type can be made. At the same time, the viewing angle information of the first-view infrared image can also be accurately obtained, laying a foundation for obtaining the three-dimensional supplementary viewing angle image, so as to improve the accurate positioning of the device fault location.

[0083] 103. If the fault category is an internal fault of the device, then obtain the second-view infrared image of the target device based on the first temperature distribution image, and determine the fault location information of the target device according to the first temperature distribution image, the image registration result of the second-view infrared image and the three-dimensional image of the device.

[0084] In the embodiments of the present invention, when a fault occurs inside the device, since the first temperature distribution image can only display the position information of the fault in the current plane and cannot display the depth information of the fault position, it is necessary to obtain an infrared image from a perspective perpendicular to the display plane of the first temperature distribution image, that is, the second perspective infrared image, to determine the depth information of the fault position. For the same reason as the first perspective infrared image, it is necessary to register the second perspective infrared image with the device three-dimensional image to obtain an image of the device structure, spatial connection relationship, and temperature distribution on the plane perpendicular to the display plane of the first temperature distribution image, and based on the device structure information, spatial connection relationship, and temperature distribution information on two mutually perpendicular planes, obtain the temperature distribution in three-dimensional space. Further, since the position of the heat source is the position of the fault point, the coordinate information of the heat source, that is, the fault position of the target device, is calculated based on the temperature distribution in three-dimensional space and the heat conduction parameters of different media. In addition, if the fault category is an external device fault, the temperature anomaly area is on the outer surface of the device, and the fault position can be directly located based on the first temperature distribution image or the first perspective infrared image.

[0085] It should be noted that by determining the second perspective perpendicular to the display plane of the first temperature distribution image and using the second perspective infrared image as a supplementary perspective image for the first perspective infrared image, the depth information that cannot be displayed in the first temperature distribution image is supplemented, and a three-dimensional space for extracting the fault position is constructed, thereby effectively improving the accuracy of device fault location. At the same time, the efficiency of device fault location is effectively improved.

[0086] In an embodiment of the present invention, for further illustration and limitation, as Figure 2 shown, step 102 of determining the first temperature distribution image of the target device by registering the first perspective infrared image with the device three-dimensional image includes:

[0087] 201. Use an image feature extraction model to extract multiple first structure features of the target device in the first perspective infrared image, and multiple second structure features of the target device in the device three-dimensional image.

[0088] 202. Calculate the pixel matching distance between the pixel regions of the first structure features and the pixel regions of the second structure features.

[0089] 203. If the pixel matching distance is less than or equal to a preset pixel matching threshold, determine the plane where the second structure feature is located as the target plane.

[0090] 204. Perform image registration processing on the first perspective infrared image and the view of the target device mapped on the target plane to obtain the first temperature distribution image.

[0091] In the embodiments of the present invention, the image feature extraction model is a convolutional neural network model pre-trained based on the marked device corner points and the marked device structure, where the marked device corner points and the marked device structure are representative structures in the device structure. Therefore, this image feature extraction model can extract the representative device corner point features and device structure features in the first-view infrared image, that is, the first structural features, as well as the representative device corner point features and device structure features in the device three-dimensional image, that is, the second structural features. Using the K-nearest neighbor algorithm, calculate the distance between the first structural features in each plane of the device three-dimensional image and the pixel regions corresponding to the second structural features in the first-view infrared image, and use this distance as the matching loss function. When the matching loss function is less than or equal to the preset pixel matching threshold, it indicates that the current plane (target plane) in the device three-dimensional image matches the first-view infrared image, that is, the first-view infrared image represents the device structure and device temperature of the target plane. Among them, the preset pixel matching threshold can be 0.1, or it can be customized according to actual needs, and the embodiments of the present invention do not make specific limitations. After determining the target plane, register the projection image of the device three-dimensional image on the target plane with the first-view infrared image to obtain a fused image of the device structure data in the device three-dimensional image and the temperature distribution data in the first-view infrared image, that is, the first temperature distribution image.

[0092] It should be noted that by pre-training the convolutional neural network model using the structural features in the device, the image feature extraction model can accurately identify the structural features in the device three-dimensional image and the first-view infrared image, and then can achieve the precise registration of the device three-dimensional image and the first-view infrared image, providing an accurate data basis for fault location, thereby improving the accuracy of fault recognition.

[0093] In an embodiment of the present invention, for further illustration and limitation, as Figure 3 shown, step 102 of determining the fault category of the target device according to the first temperature distribution image includes:

[0094] 301. Determine the region with a temperature greater than the preset abnormal temperature threshold in the first temperature distribution image as the abnormal temperature pixel region, and extract the edge irregularity and pixel ratio of the abnormal temperature pixel region.

[0095] 302. If the edge irregularity is less than or equal to the preset edge irregularity threshold and the pixel ratio is greater than or equal to the preset pixel ratio, determine that the fault category is an internal device fault.

[0096] 303. If the edge irregularity is greater than the preset edge irregularity threshold and the pixel ratio is less than the preset pixel ratio, determine that the fault category is an external device fault.

[0097] In the embodiment of the present invention, after determining the abnormal temperature pixel region in the first temperature distribution image based on the preset abnormal temperature threshold, an edge extraction tool, such as canny, is used to extract the edge of the abnormal temperature pixel region. By calculating the edge gradient distribution map, the unsigned gradient histogram is calculated, and 9 bins are statistically counted (corresponding to the 0-degree angle direction, 20-degree angle direction... 160-degree angle direction respectively). For example, if the gradient direction of the current edge point is the 60-degree angle direction and the gradient magnitude is 8, then add 8 to the bin corresponding to the 60-degree angle direction; if the gradient direction is the 55-degree angle direction and the gradient magnitude is 6, distribute the gradient magnitude proportionally to the bins corresponding to the 40-degree angle direction and the 60-degree angle direction, and distribute 4.5 and 1.5 in turn, and the distribution map of the overall histogram is statistically obtained in the obtained histogram. If the overall prominent part (greater than the distribution mean) of the distribution map is narrow (occupying three bins or less), the edge is regular. If the overall prominent part of the distribution map is wide (occupying four bins or more), the edge is irregular. The regularity is represented by the ratio of the prominent part of the distribution map to the total number of bins, and if it is greater than 1 / 3, the shape is considered irregular. It is also possible to perform statistical calculations based on the edge breakpoints to obtain the edge irregularity. For the method of calculating the edge irregularity, the embodiment of the present invention does not make specific limitations. By statistically calculating the ratio of the number of pixel points inside the edge of the abnormal temperature pixel region to the number of pixel points in the first temperature distribution image, the pixel ratio of the abnormal temperature pixel region is obtained. When the edge irregularity is relatively large and the pixel ratio is relatively small, it indicates that the current fault location is outside the device. When the edge irregularity is relatively small and the pixel ratio is relatively large, it indicates that the current fault location is inside the device. The situation where the edge irregularity is relatively small and the pixel ratio is also relatively small, and the situation where the edge irregularity is relatively large and the pixel ratio is also relatively large hardly occur. If it occurs, it indicates that the device collects abnormal data, and an instruction to re-obtain the first-view infrared image can be generated, or a request for manual intervention for confirmation can be made. Among them, the specific values of the preset abnormal temperature threshold, the preset edge irregularity threshold, and the preset pixel ratio can be customized based on application requirements, and the embodiment of the present invention does not make specific limitations.

[0098] It should be noted that if the fault location (heat source) is outside the device and the heat source is exposed to the external space, the edge of the temperature anomaly area collected by the infrared image acquisition device is relatively irregular. Moreover, since air has a worse heat conduction ability than the device material, the image area occupied by the abnormal temperature will also be relatively small. On the contrary, if the fault location (heat source) is inside the device, the influence of the device shell makes the edge of the collected temperature anomaly area relatively regular, and the image area occupied by the abnormal temperature is also relatively large. Therefore, based on the differences in the irregularity of the image edges and the proportion of image pixels when the fault point is inside and outside the device, it is possible to accurately distinguish whether the fault point is inside or outside the device, which is equivalent to a preliminary screening of the fault location, thereby effectively improving the accuracy of fault location identification.

[0099] In an embodiment of the present invention, for further illustration and limitation, step 103, when the fault category is an internal device fault, obtaining the second perspective infrared image of the target device based on the first temperature distribution image includes:

[0100] Obtain the first perspective information of the first temperature distribution image, and calculate the second perspective information according to the first perspective information.

[0101] Send a request for obtaining the second perspective infrared image to the control terminal of the infrared image acquisition device to obtain the second perspective infrared image.

[0102] In the embodiment of the present invention, the first perspective information is the coordinate information of the plane shown in the first temperature distribution image, and the second perspective information is the coordinate information in the direction perpendicular to the plane shown in the first temperature distribution image. When a device fault occurs inside the device, if only the planar temperature distribution image of the device is available, it is impossible to determine the depth direction position information of the temperature heat source (fault location). Therefore, obtain the coordinate information of the plane shown in the first temperature distribution image, and then calculate the coordinate information in the direction perpendicular to the shown plane according to the coordinate information of the shown plane. And send this coordinate information to the control terminal of the infrared image acquisition device, so that the control terminal controls or calls the corresponding infrared image acquisition device according to the coordinate information to collect the infrared image at the appropriate position in the direction perpendicular to the plane shown in the first temperature distribution image, that is, the second perspective infrared image. Among them, the infrared image acquisition device can collect images of at least two mutually perpendicular perspectives, which can be multiple devices installed at different perspectives of the device, or at least one device installed on a drone or a multi-axial moving device. The embodiment of the present invention does not make specific limitations.

[0103] In an embodiment of the present invention, for further illustration and limitation, step 103, determining the fault location information of the target device according to the feature matching result of the first temperature distribution image, the second perspective infrared image and the three-dimensional image of the device includes:

[0104] Perform image registration based on the three-dimensional image of the device and the infrared image from the second perspective to determine the second temperature distribution image of the target device.

[0105] Construct a three-dimensional temperature distribution image of the target device based on the first temperature distribution image and the second temperature distribution image.

[0106] Perform inversion calculation according to the three-dimensional temperature distribution image and the medium temperature propagation parameters to determine the fault location information of the target device.

[0107] In the embodiment of the present invention, according to the first temperature distribution image and the second temperature distribution image on the vertical plane, the coordinate information is integrated to obtain the three-dimensional coordinate information of each position on the image, and then a three-dimensional temperature distribution image including the three-dimensional position coordinate information of the temperature in the target device is obtained. Since the device material and the spatial connection relationship inside the device are known, according to the three-dimensional temperature distribution image, the heat conduction law in different media, and the temperature propagation parameters of different media, an inversion calculation of the heat source position is performed to obtain the heat source position inside the device, that is, the fault position. Among them, the implementation process of image registration between the three-dimensional image of the two devices and the infrared image from the second perspective is the same as that of step 102, and the embodiment of the present invention will not elaborate here.

[0108] It should be noted that due to the occurrence of a fault leading to an increase in temperature, the occurrence point of the fault can be determined by finding the heat source. By constructing a three-dimensional temperature distribution image, the three-dimensional coordinate information of the temperature at any point inside the device can be accurately obtained, and then the heat source position coordinate information can be determined according to the existing heat source position inversion algorithm, so as to accurately extract the three-dimensional coordinate information of the fault occurrence point in the device.

[0109] In an embodiment of the present invention, for further illustration and limitation, before the target device in step 101 is a power device and in response to the fault warning information of the target device, obtaining the first perspective infrared image of the target device and the three-dimensional image of the target device, the method further includes:

[0110] Based on the weight parameter and load parameter corresponding to each power device, determine the sampling frame number and sampling time for the infrared image acquisition device to perform image acquisition on each power device;

[0111] Generate a sampling control instruction based on the sampling frame number and send the sampling control instruction to the infrared image acquisition device to control the infrared image acquisition device to acquire the first perspective infrared image of the corresponding power device according to the sampling frame number and the sampling time;

[0112] If the maximum value of the temperature data in the first - perspective infrared image is greater than or equal to the preset fault temperature threshold, determine that the power equipment corresponding to the first - perspective infrared image is the target equipment, and generate a target - equipment warning message.

[0113] In the embodiments of the present invention, since the loss caused by a fault in a key equipment in the entire equipment system is greater, the requirement for the timeliness of fault monitoring is higher. Also, since the equipment with a larger increase in operating load compared to the conventional level is more likely to fail and also requires closer monitoring. Therefore, according to the importance level (weight parameter) and load parameter of this equipment, the sampling frame number of the infrared video to be collected is determined. Specifically, according to the importance level of different power equipment, a corresponding weight parameter is configured for the power equipment, denoted as P. For example, P = 3 represents extremely important; P = 2 represents important; P = 1 represents ordinary. According to the exceeding level of the current load rate of the power equipment relative to the historical load rate, a corresponding load parameter is configured for the power equipment, denoted as T. For example, if the current operating load rate of the equipment exceeds the historical average load rate by 60%, then the load parameter is determined to be 3; if the current operating load rate of the equipment exceeds the historical average load rate by 30%, then the load parameter is determined to be 2; if the current operating load rate of the equipment is less than or equal to the historical average load rate, then the load parameter is determined to be 1. The sampling frame number n is determined according to the load parameter and the weight parameter, n = K×(P + T) (1); where K is the sampling frame rate ratio and can take 0.5 (30 frames / 60 seconds). The sampling time is the product of the sampling frame number and the preset time coefficient. For example, if the sampling frame number is 4.5 and the preset time coefficient is 180s, then the sampling time is 810s. Among them, the preset time coefficient can be customized according to actual needs, and the embodiments of the present invention do not make specific limitations.

[0114] It should be noted that the infrared - image acquisition device is generally a mobile device, and it is necessary to record the infrared videos of different power equipment by traveling between multiple power equipment to realize the monitoring of the equipment. Based on the weight parameters and load parameters of different power equipment, the sampling attention degree to be given to the power equipment, that is, the sampling time and sampling frame number, can ensure the key attention and monitoring of important equipment or important operation stages of the equipment, while reducing the attention and monitoring of unimportant equipment, reducing the work burden, and realizing the intelligent allocation of the infrared - image acquisition device, achieving full - coverage and key monitoring of equipment monitoring, thereby effectively improving the efficiency of equipment monitoring.

[0115] In an embodiment of the present invention, for further illustration and limitation, after step 103 of determining the fault location information of the target equipment according to the first temperature - distribution image and the feature - matching result between the second - perspective infrared image and the equipment three - dimensional image, the method further includes:

[0116] Extracting abnormal temperature and spatial position information corresponding to the fault location, and a component structure image of the device component where the fault location is located;

[0117] Determining a fault state based on a comparison result of the abnormal temperature with a plurality of preset temperature thresholds, wherein the preset temperature thresholds correspond to fault states representing different fault degrees;

[0118] Using an image semantic segmentation model to perform semantic recognition on the component structure image to obtain component semantic information of the faulty device component;

[0119] Equipment maintenance information is generated based on the component semantic information and the spatial position information, and the equipment maintenance information is output to a display terminal so that maintenance personnel can repair or replace the equipment component.

[0120] In an embodiment of the present invention, after determining the location of the equipment fault, it is also necessary to repair or replace the faulty equipment component. In this process, it is necessary to use computer equipment to retrieve the equipment structure, and it is also necessary to perform repair or replacement operations by the current staff. In order to achieve collaborative understanding between the staff and the computer equipment, as well as equipment configuration retrieval, corresponding standardized semantic information is constructed for the faulty components in the equipment. The equipment maintenance information includes the equipment name, the faulty component name, the spatial location information of the faulty component, the temperature anomaly, and the fault state. Among them, the fault state is the standardized semantic information that characterizes the degree of temperature anomaly, such as critical, urgent, etc. And the number of bytes of each semantic information is specifically limited, for example, the number of bytes of the equipment name is less than 10, and the number of bytes of the fault state is less than 6. For example, a 220kV transformer has a heating fault due to internal magnetic leakage, short circuit, and circulating current. The equipment maintenance information includes: 220kV transformer-internal components (such as iron core, winding, etc.)-(x, y, z) coordinates-80℃-critical. Among them, the fault state corresponds to one of multiple abnormal temperature levels, and each abnormal temperature level corresponds to a preset temperature threshold. For example, the first preset temperature threshold is 100°C, corresponding to fault state 3; the second preset temperature threshold is 80, corresponding to fault state 2; the third preset temperature threshold is 60, corresponding to fault state 1. The fault component name is the component semantic information of the device component where the fault location is located, which is obtained by semantically segmenting the component structure image of the device component where the fault location is located based on the image semantic segmentation model.

[0121] It should be noted that by constructing the normalized semantic information of device components, fault status, and spatial location information, and generating device maintenance information, it is possible to enable staff to more conveniently confirm faulty device components and find suitable device components for maintenance, thereby effectively improving the fault repair efficiency. In addition, after constructing the normalized semantic information, it is also possible to retrieve the 3D models of device components based on the normalized semantic information of device components. Since the cost of building the actual 3D structure is relatively high, and some component structures of power equipment are universal. For example, insulators can use a method similar to BIM to assist in constructing a 3D model database of devices based on the existing 3D models of devices, so as to achieve the mutual matching and rapid call of the same structures of different devices.

[0122] The present invention provides a method for locating device faults. In an embodiment of the present invention, in response to the fault warning information of a target device, a first perspective infrared image of the target device and a device 3D image of the target device are acquired; by performing image registration on the first perspective infrared image and the device 3D image, a first temperature distribution image of the target device is determined, and the fault category of the target device is determined according to the first temperature distribution image; if the fault category is an internal device fault, a second perspective infrared image of the target device is acquired based on the first temperature distribution image, and the fault location information of the target device is determined according to the first temperature distribution image and the feature matching results of the second perspective infrared image and the device 3D image, which can greatly reduce the dependence on manual labor in device fault location, avoid the inaccuracy of manual experience judgment, achieve precise positioning of the fault location, and thus effectively improve the accuracy of power device fault location while reducing labor costs.

[0123] Further, as an implementation of the method described above Figure 1 shown, an embodiment of the present invention provides a device for locating device faults, as Figure 4 shown, the device includes:

[0124] An acquisition module 41, configured to acquire a first perspective infrared image of the target device and a device 3D image of the target device in response to the fault warning information of the target device;

[0125] A first determination module 42, configured to determine a first temperature distribution image of the target device by performing image registration on the first perspective infrared image and the device 3D image, and determine the fault category of the target device according to the first temperature distribution image;

[0126] The second determination module 43 is configured to, if the fault category is an internal fault of the device, obtain a second perspective infrared image of the target device based on the first temperature distribution image, and determine the fault location information of the target device according to the first temperature distribution image, and the feature matching result between the second perspective infrared image and the three-dimensional image of the device.

[0127] Further, the first determination module includes:

[0128] The first extraction unit is configured to extract a plurality of first structural features of the target device in the first perspective infrared image and a plurality of second structural features of the target device in the three-dimensional image of the device by using an image feature extraction model;

[0129] The calculation unit is configured to calculate the pixel matching distance between the pixel region of the first structural feature and the pixel region of the second structural feature;

[0130] The first determination unit is configured to, if the pixel matching distance is less than or equal to a preset pixel matching threshold, determine the plane where the second structural feature is located as the target plane;

[0131] The registration unit is configured to perform image registration processing on the first perspective infrared image and the view of the target device mapped on the target plane to obtain a first temperature distribution image.

[0132] Further, the first determination module further includes:

[0133] The second extraction unit is configured to determine the region where the temperature in the first temperature distribution image is greater than a preset abnormal temperature threshold as an abnormal temperature pixel region, and extract the edge irregularity and the pixel ratio of the abnormal temperature pixel region, where the pixel ratio is the pixel ratio of the abnormal temperature pixel region in the first temperature distribution image;

[0134] The second determination unit is configured to, if the edge irregularity is less than or equal to a preset edge irregularity threshold and the pixel ratio is greater than or equal to a preset pixel ratio, determine that the fault category is an internal fault of the device;

[0135] The third determination unit is configured to, if the edge irregularity is greater than a preset edge irregularity threshold and the pixel ratio is less than a preset pixel ratio, determine that the fault category is an external fault of the device.

[0136] Further, the second determination module includes:

[0137] A first acquisition unit, configured to acquire first perspective information of the first temperature distribution image, and calculate second perspective information according to the first perspective information, where the second perspective information represents a perspective perpendicular to a plane corresponding to the first perspective information;

[0138] A second acquisition unit, configured to acquire a second perspective infrared image by sending a second perspective infrared image acquisition request to a control terminal of an infrared image acquisition device, where the second perspective infrared image acquisition request is generated based on the second perspective information.

[0139] Further, the second determination module includes:

[0140] A fourth determination unit, configured to perform image registration according to the device three-dimensional image and the second perspective infrared image to determine a second temperature distribution image of the target device;

[0141] A construction unit, configured to construct a three-dimensional temperature distribution image of the target device based on the first temperature distribution image and the second temperature distribution image;

[0142] A fifth determination unit, configured to perform inversion calculation according to the three-dimensional temperature distribution image and a medium temperature propagation parameter to determine fault location information of the target device.

[0143] Further, the apparatus further includes:

[0144] The determination module is further configured to determine a sampling frame number and a sampling time for the infrared image acquisition device to perform image acquisition on each power device based on a weight parameter and a load parameter corresponding to each power device;

[0145] A generation module, configured to generate a sampling control instruction based on the sampling frame number and the sampling time, and send the sampling control instruction to the infrared image acquisition device to control the infrared image acquisition device to acquire a first perspective infrared image of a corresponding power device according to the sampling frame number and the sampling time;

[0146] An early warning module, configured to, if a maximum value of temperature data in the first perspective infrared image is greater than or equal to a preset fault temperature threshold, determine the power device corresponding to the first perspective infrared image as a target device, and generate target device early warning information.

[0147] Further, the apparatus further includes:

[0148] An extraction module, configured to extract abnormal temperature, spatial position information corresponding to the fault location, and a component structure image of a device component where the fault location is located;

[0149] The determining module is further configured to determine a fault status based on a comparison result between the abnormal temperature and a plurality of preset temperature thresholds, where the preset temperature thresholds correspond to fault statuses representing different degrees of faults;

[0150] The semantic segmentation module is configured to perform semantic recognition on the component structure image by using an image semantic segmentation model to obtain component semantic information of the faulty device component;

[0151] The generating module is further configured to generate device maintenance information based on the component semantic information and the spatial position information, and output the device maintenance information to a display terminal so that maintenance personnel can repair or replace the device component.

[0152] The present invention provides a device fault location device. In an embodiment of the present invention, by responding to a fault warning message of a target device, a first perspective infrared image of the target device and a device three-dimensional image of the target device are obtained; by performing image registration on the first perspective infrared image and the device three-dimensional image, a first temperature distribution image of the target device is determined, and a fault category of the target device is determined according to the first temperature distribution image; if the fault category is an internal device fault, a second perspective infrared image of the target device is obtained based on the first temperature distribution image, and a fault location information of the target device is determined according to the first temperature distribution image and a feature matching result between the second perspective infrared image and the device three-dimensional image, which can greatly reduce the dependence on manual labor in device fault location, avoid the inaccuracy of manual experience judgment, achieve precise positioning of the fault location, thereby reducing the labor cost while effectively improving the accuracy of power device fault location.

[0153] According to an embodiment of the present invention, a storage medium is provided, and the storage medium stores at least one executable instruction, and the computer executable instruction can execute the device fault location method in any of the above method embodiments.

[0154] Figure 5 FIG. shows a schematic structural diagram of a terminal according to an embodiment of the present invention. The specific implementation of the terminal is not limited in the specific embodiment of the present invention.

[0155] As Figure 5 shown, the terminal may include: a processor 502, a communication interface 504, a memory 506, and a communication bus 508.

[0156] Wherein: the processor 502, the communication interface 504, and the memory 506 communicate with each other through the communication bus 508.

[0157] A communication interface 504 for communicating with network elements of other devices such as clients or other servers.

[0158] A processor 502 for executing a program 510, which can specifically execute the relevant steps in the embodiments of the above device fault location method.

[0159] Specifically, the program 510 may include program code, and the program code includes computer operation instructions.

[0160] The processor 502 may be a central processing unit (CPU), or a specific integrated circuit (ASIC) (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention. One or more processors included in the terminal may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0161] A memory 506 for storing the program 510. The memory 506 may include high-speed RAM memory and may also include non-volatile memory, such as at least one disk memory.

[0162] The program 510 can specifically be used to cause the processor 502 to perform the following operations:

[0163] In response to the fault warning information of the target device, obtain the first perspective infrared image of the target device and the three-dimensional image of the target device;

[0164] By performing image registration on the first perspective infrared image and the three-dimensional image of the device, determine the first temperature distribution image of the target device, and determine the fault category of the target device according to the first temperature distribution image;

[0165] If the fault category is an internal device fault, obtain the second perspective infrared image of the target device based on the first temperature distribution image, and determine the fault location information of the target device according to the first temperature distribution image and the feature matching result between the second perspective infrared image and the three-dimensional image of the device.

[0166] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present invention can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. Optionally, they can be implemented by program codes executable by the computing device. Thus, they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a sequence different from that here, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. In this way, the present invention is not limited to any specific combination of hardware and software.

[0167] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for locating equipment faults, characterized in that, Including: In response to the fault warning information of the target device, obtaining the first - perspective infrared image of the target device and the three - dimensional device image of the target device; By performing image registration on the first - perspective infrared image and the three - dimensional device image, determining the first temperature distribution image of the target device, and determining the fault category of the target device according to the first temperature distribution image, wherein the first temperature distribution image is an image containing device structure information, spatial connection relationship, and temperature distribution information under the current perspective; If the fault category is an internal device fault, based on the first temperature distribution image, obtaining the second - perspective infrared image of the target device, and determining the fault location information of the target device according to the first temperature distribution image and the feature matching result between the second - perspective infrared image and the three - dimensional device image; Among them, determining the fault category of the target device according to the first temperature distribution image includes: Determining the region where the temperature in the first temperature distribution image is greater than the preset abnormal temperature threshold as the abnormal temperature pixel region, and extracting the edge irregularity and pixel ratio of the abnormal temperature pixel region, where the pixel ratio is the pixel ratio of the abnormal temperature pixel region in the first temperature distribution image; If the edge irregularity is less than or equal to the preset edge irregularity threshold and the pixel ratio is greater than or equal to the preset pixel ratio, determining that the fault category is an internal device fault; If the edge irregularity is greater than the preset edge irregularity threshold and the pixel ratio is less than the preset pixel ratio, determining that the fault category is an external device fault.

2. The method according to claim 1, wherein The method of determining the first temperature distribution image of the target device by performing image registration on the first - perspective infrared image and the three - dimensional device image includes: Using an image feature extraction model to extract multiple first structure features of the target device in the first - perspective infrared image and multiple second structure features of the target device in the three - dimensional device image; Calculating the pixel matching distance between the pixel region of the first structure feature and the pixel region of the second structure feature; If the pixel matching distance is less than or equal to the preset pixel matching threshold, determining that the plane where the second structure feature is located is the target plane; Performing image registration processing on the first - perspective infrared image and the view of the target device mapped on the target plane to obtain the first temperature distribution image.

3. The method according to claim 1, wherein The step of, if the fault category is an internal device fault, obtaining the second - perspective infrared image of the target device based on the first temperature distribution image includes: Obtaining the first - perspective information of the first temperature distribution image, and calculating the second - perspective information according to the first - perspective information, where the second - perspective information represents the perspective perpendicular to the plane corresponding to the first - perspective information; Sending a second - perspective infrared image acquisition request to the control terminal of the infrared image acquisition device, and obtaining the second - perspective infrared image, where the second - perspective infrared image acquisition request is generated based on the second - perspective information.

4. The method according to claim 1, wherein Determining the fault location information of the target device according to the first temperature distribution image, and the feature matching result between the second perspective infrared image and the three-dimensional image of the device includes: Performing image registration on the three-dimensional image of the device and the second perspective infrared image to determine the second temperature distribution image of the target device; Constructing a three-dimensional temperature distribution image of the target device based on the first temperature distribution image and the second temperature distribution image; Performing inversion calculation according to the three-dimensional temperature distribution image and the medium temperature propagation parameters to determine the fault location information of the target device.

5. The method according to claim 1, wherein Before the target device is an electrical device, and in response to the fault warning information of the target device, obtaining the first perspective infrared image of the target device and the three-dimensional image of the target device, the method further includes: Based on the weight parameter and load parameter corresponding to each electrical device, determining the sampling frame number and sampling time for the infrared image acquisition device to perform image acquisition on each electrical device; Generating a sampling control instruction based on the sampling frame number and the sampling time, and sending the sampling control instruction to the infrared image acquisition device to control the infrared image acquisition device to acquire the first perspective infrared image of the corresponding electrical device according to the sampling frame number and the sampling time; If the maximum value of the temperature data in the first perspective infrared image is greater than or equal to a preset fault temperature threshold, determining the electrical device corresponding to the first perspective infrared image as the target device, and generating a target device warning information.

6. The method according to claim 1, characterized in that After determining the fault location information of the target device according to the first temperature distribution image, and the feature matching result between the second perspective infrared image and the three-dimensional image of the device, the method further includes: Extracting the abnormal temperature, spatial location information corresponding to the fault location, and the component structure image of the device component where the fault location is located; Determining the fault state based on the comparison result between the abnormal temperature and a plurality of preset temperature thresholds, where the preset temperature thresholds correspond to the fault states representing different fault degrees; Performing semantic recognition on the component structure image by using an image semantic segmentation model to obtain the component semantic information of the device component; Generating device maintenance information based on the component semantic information and the spatial location information, and outputting the device maintenance information to a display terminal so that maintenance personnel can perform maintenance or replacement on the device component.

7. A device fault location device, characterized in that, Including: An acquisition module, configured to obtain the first perspective infrared image of the target device and the three-dimensional image of the target device in response to the fault warning information of the target device; A first determination module, configured to determine the first temperature distribution image of the target device by performing image registration on the first perspective infrared image and the three-dimensional image of the device, and determine the fault category of the target device according to the first temperature distribution image, where the first temperature distribution image is an image including device structure information, spatial connection relationship, and temperature distribution information in the current perspective; A second determination module, configured to, if the fault category is an internal fault of the device, obtain a second perspective infrared image of the target device based on the first temperature distribution image, and determine the fault location information of the target device according to the first temperature distribution image, and the feature matching result between the second perspective infrared image and the three-dimensional image of the device; Wherein, the first determination module further includes: A second extraction unit, configured to determine an abnormal temperature pixel region in the first temperature distribution image where the temperature is greater than a preset abnormal temperature threshold, and extract the edge irregularity and pixel ratio of the abnormal temperature pixel region, where the pixel ratio is the pixel ratio of the abnormal temperature pixel region in the first temperature distribution image; A second determination unit, configured to determine that the fault category is an internal fault of the device if the edge irregularity is less than or equal to a preset edge irregularity threshold and the pixel ratio is greater than or equal to a preset pixel ratio; A third determination unit, configured to determine that the fault category is an external fault of the device if the edge irregularity is greater than a preset edge irregularity threshold and the pixel ratio is less than a preset pixel ratio.

8. A storage medium, in which at least one executable instruction is stored, and the executable instruction causes a processor to perform operations corresponding to the device fault location method according to any one of claims 1-6.

9. A terminal, comprising: A processor, a memory, a communication interface, and a communication bus, where the processor, the memory, and the communication interface complete communication with each other through the communication bus; The memory is used to store at least one executable instruction, and the executable instruction causes the processor to perform operations corresponding to the device fault location method according to any one of claims 1-6.

Images