Intelligent detection method, device and electronic equipment based on temperature vision

Abstract

The present invention proposes an intelligent detection method, device and electronic device based on temperature vision, which relates to the field of computer vision technology. The method includes: obtaining a temperature visual image of a target power device; inputting the temperature visual image into a pre-trained target detection model to obtain the type information of the target power device; determining a target area image and a sub-classification model corresponding to the target power device according to the type information and location information of the target power device; inputting the target area image into the sub-classification model to obtain an image to be detected containing each component box marked in the target area image; inputting the image to be detected into a pre-trained fault detection model to determine the fault type and fault location of the target power device. Thus, based on the temperature visual information, the position of the power equipment can be located through the target detection network, and the position of the power equipment components can be located through the feature point network, so as to accurately locate the positions of the components of the power equipment and obtain the corresponding temperature information.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an intelligent detection method, device and electronic equipment based on temperature vision. Background Art

[0002] At present, the detection method of power equipment is mainly based on infrared images to study the problem of abnormal heating of power equipment. Since infrared images contain multiple pseudo-colors, the complex shooting environment causes serious interference from the photographed equipment, and there are few types of training data, the existing methods have defects such as low detection accuracy and poor model generalization ability.

[0003] Therefore, how to accurately and reliably detect power equipment is a problem that needs to be solved urgently. Summary of the invention

[0004] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0005] The first embodiment of the present invention proposes an intelligent detection method based on temperature vision, comprising:

[0006] Acquire a visual image of the temperature of the target power equipment;

[0007] Inputting the temperature visual image into a pre-trained target detection model to obtain type information of the target power equipment;

[0008] Determine a target area image and a sub-classification model corresponding to the target power equipment according to the type information and location information of the target power equipment, including: cropping the temperature visual image of the target power equipment according to the location information of the target power equipment to obtain the target area image, and determining the sub-classification model corresponding to the current target power equipment based on the correspondence between the type information of the target power equipment and the sub-classification model;

[0009] Inputting the target area image into the sub-classification model to obtain an image to be detected containing each component frame marked in the target area image;

[0010] The image to be detected is input into a pre-trained fault detection model to determine the fault type and fault location of the target power equipment.

[0011] Optionally, before inputting the training data set into the pre-trained target detection model to obtain the type information and location information of the target power equipment, the method further includes:

[0012] Acquire a pre-trained data set of a target power device, wherein the pre-trained data set is a temperature visual data set of the target power device;

[0013] Annotating the temperature visual data set in a specified manner, wherein the specified manner at least includes device frame annotation and feature point annotation;

[0014] The labeled temperature vision dataset is used to train the initial detection model to obtain the target detection model.

[0015] Optionally, before determining the target area image and the sub-classification model corresponding to the target power equipment according to the type information and location information of the target power equipment, the method further includes:

[0016] Crop each temperature visual image in the pre-training data set of the target power equipment to obtain a training data set containing the labeled area;

[0017] Classifying the training data set according to the type of target power equipment to obtain multiple training data subsets;

[0018] The multiple training data subsets are used to train multiple corresponding initial sub-classification models to obtain multiple trained sub-classification models.

[0019] Optionally, before inputting the image to be detected into a pre-trained fault detection model, the method further includes:

[0020] Obtain a binary temperature data set divided into faulty devices and non-faulty devices;

[0021] The initial fault diagnosis model is trained using the binary temperature data set to generate a fault detection model.

[0022] Optionally, also include:

[0023] Inputting the image to be detected into the fault detection model generated by training to obtain the annotation label corresponding to the image to be detected, wherein the annotation label includes a fault type label and a fault location label;

[0024] Inputting the image to be detected into the initial fault diagnosis model to be trained to obtain a prediction label corresponding to the image to be detected;

[0025] According to the difference between the predicted label and the marked label, the initial fault diagnosis model and the fault detection model are respectively modified.

[0026] The second embodiment of the present invention proposes an intelligent detection device based on temperature vision, comprising:

[0027] A first acquisition module is used to acquire a temperature visual image of a target power device;

[0028] A second acquisition module is used to input the temperature visual image into a pre-trained target detection model to obtain type information of the target power equipment;

[0029] A first determination module is used to determine a target area image and a sub-classification model corresponding to the target power equipment according to the type information and location information of the target power equipment, and is specifically used to: crop the temperature visual image of the target power equipment according to the location information of the target power equipment to obtain the target area image, and determine the sub-classification model corresponding to the current target power equipment based on the correspondence between the type information of the target power equipment and the sub-classification model;

[0030] A third acquisition module is used to input the target area image into the sub-classification model to obtain an image to be detected containing each component frame marked in the target area image;

[0031] The second determination module is used to input the image to be detected into a pre-trained fault detection model to determine the fault type and fault location of the target power equipment.

[0032] Optionally, the second acquisition module is further used to:

[0033] Acquire a pre-trained data set of a target power device, wherein the pre-trained data set is a temperature visual data set of the target power device;

[0034] Annotating the temperature visual data set in a specified manner, wherein the specified manner at least includes device frame annotation and feature point annotation;

[0035] The labeled temperature vision dataset is used to train the initial detection model to obtain the target detection model.

[0036] Optionally, the first determining module is further used to:

[0037] Crop each temperature visual image in the pre-training data set of the target power equipment to obtain a training data set containing the labeled area;

[0038] Classifying the training data set according to the type of target power equipment to obtain multiple training data subsets;

[0039] The multiple training data subsets are used to train multiple corresponding initial sub-classification models to obtain multiple trained sub-classification models.

[0040] Optionally, the second determining module is further used to:

[0041] Obtain a binary temperature data set divided into faulty devices and non-faulty devices;

[0042] The initial fault diagnosis model is trained using the binary temperature data set to generate a fault detection model.

[0043] Optionally, the device further comprises:

[0044] A third acquisition module is used to input the image to be detected into the fault detection model generated by training to obtain a label corresponding to the image to be detected, wherein the label includes a fault type label and a fault location label;

[0045] A fourth acquisition module, used for inputting the image to be detected into the initial fault diagnosis model to be trained to obtain a prediction label corresponding to the image to be detected;

[0046] The correction module is used to correct the initial fault diagnosis model and the fault detection model respectively according to the difference between the predicted label and the marked label.

[0047] The third aspect of the present invention provides a non-temporary computer-readable storage medium storing a computer program, which, when executed by a processor, implements the intelligent detection method based on temperature vision as provided in the first aspect of the present invention.

[0048] The fourth aspect of the present invention provides a computer program product. When the instruction processor in the computer program product executes, the intelligent detection method based on temperature vision proposed in the first aspect of the present invention is executed.

[0049] In an embodiment of the present invention, a temperature visual image of a target power device is first obtained, and then the temperature visual image is input into a pre-trained target detection model to obtain type information of the target power device. Then, based on the type information and location information of the target power device, a target area image and a sub-classification model corresponding to the target power device are determined. Then, the target area image is input into the sub-classification model to obtain an image to be detected that contains each component box marked in the target area image. Finally, the image to be detected is input into a pre-trained fault detection model to determine the fault type and fault location of the target power device. Thus, the temperature visual data can be used to train the target detection model to obtain the type and location of the device in the temperature visual image, the component location of the power device in the image can be locked more accurately, and the fault of the power device can be intelligently diagnosed.

[0050] That is, based on temperature visual information, the position of power equipment can be located through the target detection network, and the position of power equipment components can be located through the feature point network, so as to accurately locate the position of each component of the power equipment and obtain the corresponding temperature information.

[0051] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A schematic diagram of a flow chart of an intelligent temperature detection method based on vision provided by an embodiment of the present invention;

[0053] Figure 2 A schematic diagram of the structure of an intelligent temperature detection device based on vision provided by an embodiment of the present invention;

[0054] Figure 3 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present invention is shown. DETAILED DESCRIPTION

[0055] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0056] The following describes the intelligent detection method, device and electronic device based on temperature vision according to the embodiments of the present invention with reference to the accompanying drawings. The intelligent detection method based on temperature vision according to the embodiments of the present invention can be performed by the intelligent detection device based on temperature vision provided by the embodiments of the present invention, which can be configured in an electronic device.

[0057] For the convenience of explanation, the temperature vision-based intelligent detection device may be referred to as “the device”.

[0058] Figure 1 The figure is a flow chart of the intelligent detection method based on temperature vision provided by the embodiment of the present invention. Figure 1 As shown, the intelligent detection method based on temperature vision may include the following steps:

[0059] Step 101, obtaining a temperature visual image of a target power device.

[0060] Specifically, an infrared thermal imager may be used to collect information of the target power equipment to obtain a temperature visual image of the target power equipment.

[0061] Optionally, a pre-training data set of a target power device may be first acquired, wherein the pre-training data set is a temperature visual data set of the target power device.

[0062] Among them, the target power equipment can be the equipment to be diagnosed currently, which can include various types of power equipment. For example, it can include GIS air chamber, neutral point voltage transformer, JP cabinet, bushing, low voltage pile head, damping resistor box, combined filter, high voltage side bushing, lighting box, metal wire, incoming line warehouse, gathering box, pump, insulator, gathering box, GIS bushing, combined mutual inductor, transfer box, lightning arrester, coupling mutual inductor, junction box, transformer, battery pack, mechanism box, switch cabinet, filter, power box, wall bushing, lightning rod, terminal cabinet, magnetic reactor, busbar, capacitor box, station transformer, busbar, resistor box, low voltage side pile head, supporting insulator, metering box, reactor, pole-mounted vacuum switch, intelligent control cabinet, cable terminal, coupling capacitor, intelligent control cabinet, power cable, air switch, intelligent component cabinet, current Transformers, compensation devices, intelligent terminal cabinets, capacitors, fuses, panel cabinets, oil tanks, vertical insulators, maintenance power boxes, arc extinguishing devices, conductors, maintenance power line boxes, voltage transformers, radiators, capacitor boxes, connecting pipes, wire clamps, control boxes, resistors, arc extinguishing coils, control cabinets, drop-out fuses, suspension insulators, distribution boxes, connecting bridges, oil pillows, terminal boxes, soft connections, low-resistance, damping boxes, circuit breakers, inflatable bushings, regulating cabinets, discharge gaps, power boxes, discharge coils, pole-mounted circuit breakers, electric control boxes, voltage dividers, pole-mounted disconnectors, component cabinets, cable terminal tail pipes, pile heads, switch boxes, high-voltage fuses, wave arresters, wind control boxes, high-voltage bushings, pipe mothers, low-voltage switch boxes, disconnectors, AC filters, low-voltage side switch boxes, converter valves, grounding transformers, maintenance boxes and cold control boxes are not limited here.

[0063] Specifically, an infrared thermal imager can be used to collect information on the target power equipment to obtain an infrared general data file storage format image of the target power equipment, that is, an infrared thermal image, and then extract a temperature visual image from it as a temperature visual data set, that is, a pre-training data set.

[0064] It should be noted that the infrared temperature sensor can reflect the infrared radiation energy distribution diagram of the target received by the detector to the photosensitive element of the infrared detector to obtain a file in the infrared general data file storage format processed by the processor, that is, an infrared thermal image.

[0065] Among them, the infrared thermal image may include the following data formats, such as: file header, calibration data, temperature measurement parameters, temperature data, imaging parameters, analysis data, sound and user-defined data and file footer, etc., which are not limited here.

[0066] Among them, the file header can be the file version, image resolution, and image shooting time. The calibration data can be the calibration data and script data stored in the flash and written during calibration. The temperature measurement parameters can be some parameters affecting temperature measurement such as storage emissivity, ambient temperature, relative humidity, and distance. The imaging parameters can be imaging range and isothermal analysis data. The file tail can be the offset address (for fast positioning) of four data segments such as file header, temperature measurement parameters, temperature data, and sound data, file identification and other data, which are not limited here.

[0067] For example, in the corresponding imaging, the temperature data part can be a two-dimensional matrix of the fl oat type with the same length and width as the resolution of the infrared thermal image, and the ambient temperature is imaged in the form of numerical values, which is temperature vision. In some cases, the value of an element in the matrix can correspond to the actual temperature value of an area in the environment. For example, if the temperature value in the upper left corner of the matrix is ​​37.0, the actual temperature of the upper left corner area in the corresponding environment is 37.0℃.

[0068] Step 102: Input the temperature visual image into a pre-trained target detection model to obtain type information and location information of the target power equipment.

[0069] It should be noted that before inputting the training data set into the target detection model, the original input training data set needs to be standardized to reduce the impact of large differences between the data on the results.

[0070] The target detection model may be R-CNN, Fast R-CNN, Faster R-CNN, YOLO, SSD, RetinaNet, RefineDet, etc. It should be noted that in step 102, the pre-training data set is annotated in a specified manner to generate a training data set, wherein the specified manner at least includes device box annotation and feature point annotation.

[0071] Optionally, after obtaining the pre-trained data set, the temperature vision data set is annotated in a specified manner, wherein the specified manner includes at least device box annotation and feature point annotation, and then the annotated temperature vision data set is used to train the initial detection model to obtain a target detection model.

[0072] The designated method may be a predetermined method, such as equipment frame labeling, that is, labeling the target frame according to the region of interest. In addition, feature points in the power equipment may also be labeled, which is not limited here.

[0073] It should be noted that the initial detection model can be trained through the labeled temperature visual images to obtain the target detection model.

[0074] Step 103, determining the target area image and the sub-classification model corresponding to the target power equipment based on the type information and location information of the target power equipment, including: cropping the temperature visual image of the target power equipment according to the location information of the target power equipment to obtain the target area image, and determining the sub-classification model corresponding to the current target power equipment based on the correspondence between the preset type information of the target power equipment and the sub-classification model.

[0075] As a possible implementation method, each temperature visual image in a pre-trained data set of the target power equipment can be first cropped to obtain a training data set containing annotated areas, and then the training data set can be classified according to the type of the target power equipment to obtain multiple training data subsets. Thereafter, multiple corresponding initial sub-classification models can be trained using the multiple training data subsets to obtain multiple trained sub-classification models.

[0076] By cropping, the background part of the temperature visual image except the marked box can be removed, thereby leaving only the area of ​​interest. By classifying the training data set, a training data subset corresponding to each type of power equipment can be obtained, which is not limited here.

[0077] The initial sub-classification model may be a network model currently to be trained, and the sub-classification model may be a network model that has been trained.

[0078] Through the sub-classification model, the cropped temperature visual image, that is, the target area image, can be recognized, so that the boxes of multiple component positions in the target area image, that is, the image box, can be obtained.

[0079] Optionally, the temperature visual image of the target power equipment can be cropped according to the location information of the target power equipment to obtain the target area image, and then the sub-classification model corresponding to the current target power equipment can be determined based on the correspondence between the preset type information of the target power equipment and the sub-classification model.

[0080] It should be noted that, in order to more accurately diagnose the power equipment, the present invention can detect the location of the components in each power equipment based on the temperature visual image, and then perform fault diagnosis on the power equipment. In order to more accurately obtain the location of each component in the power equipment, the present invention performs the power equipment component detection in steps.

[0081] For various power equipment, since each type of power equipment has different component structures, appearance shapes, etc., first, each power equipment is classified into multiple small categories. Then, a convolutional neural network is used to crop the labeled box area of ​​the temperature visual target box annotation dataset of the above multiple power equipment, and then the obtained power equipment temperature visual dataset, that is, the data with the background removed, is trained, so that multiple classification models can be obtained to classify different appearance devices under the same type of equipment.

[0082] It can be understood that since the temperature visual data obtains the device location information and type information through the target detection model, the temperature visual data of the device area is cropped according to the device location information, and then the corresponding classification model can be called for classification according to the type information.

[0083] Specifically, each type of power equipment can be divided more finely according to appearance, that is, divided into small categories, and then multiple small category models can be obtained.

[0084] Step 104 , input the target area image into the sub-classification model to obtain an image to be detected that contains each component frame marked in the target area image.

[0085] It should be noted that for each small classification of power equipment, there is a corresponding sub-classification model. For example, it can be based on LeNet, AlexNet, CaffeNet, ZFNet, VggNet, ResNet, N i N, GoogLeNet, DenseNet, Senet, BAM, MobileNet and other convolutional neural networks, which can be modified into feature point extraction networks. The network outputs the feature point coordinates of each component of the power equipment, and trains the temperature visual data set marked with feature points to obtain a feature point extraction model. The device can detect the image to be detected of the power equipment according to the sub-classification model, output the location of the feature points of the power equipment, and finally convert the feature points into component frames to frame the positions of the components of the power equipment for subsequent fault diagnosis.

[0086] Step 105 , input the image to be detected into a pre-trained fault detection model to determine the fault type and fault location of the target power equipment.

[0087] Optionally, a binary temperature data set divided according to faulty devices and non-faulty devices may be first obtained, and then the initial fault diagnosis model may be trained using the binary temperature data set to generate a fault detection model.

[0088] Specifically, for a given power equipment, it can be pre-divided into a binary temperature vision data set according to faulty equipment and non-faulty equipment, and then the initial fault diagnosis models such as LeNet, AlexNet, VggNet, ResNet, Nin, GoogLeNet, DenseNet, Senet, BAM, MobileNet, etc. are used to train this binary temperature vision data set to obtain a fault diagnosis model. The fault diagnosis model can detect the output temperature vision data and determine whether it is a fault or not, thus realizing intelligent diagnosis of temperature vision.

[0089] Optionally, the server can also input the image to be detected into the fault detection model generated by training to obtain the annotation labels corresponding to the image to be detected, wherein the annotation labels include fault type labels and fault location labels, and then input the image to be detected into the initial fault diagnosis model to be trained to obtain the predicted labels corresponding to the image to be detected. Finally, according to the difference between the predicted labels and the annotation labels, the initial fault diagnosis model and the fault detection model are respectively corrected.

[0090] Specifically, the device can compare the predicted label with the labeled label to determine the difference between the predicted label and the labeled label. For example, the correction gradient can be determined by using gradient descent, stochastic gradient descent, etc., and then the initial fault diagnosis model and the fault detection model generated by training are corrected respectively. The present invention does not limit the method for determining the correction gradient.

[0091] Optionally, the parameters of the initial fault diagnosis model can be modified by using a regression network through the cross loss of the fault detection model, so that the fault detection model can output the fault corresponding to the current image to be detected more accurately and reliably.

[0092] It should be noted that the present invention can accurately identify and locate 106 types of power equipment based on temperature visual image data, with mAP = 98% for identifying the types of 106 types of power equipment, mAP = 99% for identifying small categories of 106 types of power equipment, IoU = 95% for locating feature points of components of 106 types of power equipment, and acc = 97% for fault diagnosis of 106 types of power equipment.

[0093] In an embodiment of the present invention, a temperature visual image of a target power device is first obtained, and then the temperature visual image is input into a pre-trained target detection model to obtain type information of the target power device. Then, based on the type information and location information of the target power device, a target area image and a sub-classification model corresponding to the target power device are determined. Then, the target area image is input into the sub-classification model to obtain an image to be detected that contains each component box marked in the target area image. Finally, the image to be detected is input into a pre-trained fault detection model to determine the fault type and fault location of the target power device. Thus, the temperature visual data can be used to train the target detection model to obtain the type and location of the device in the temperature visual image, the component location of the power device in the image can be locked more accurately, and the fault of the power device can be intelligently diagnosed.

[0094] Based on temperature visual information, the position of power equipment is located through the target detection network, and the position of power equipment components is located through the feature point network, so as to accurately locate the position of each component of the power equipment and obtain the corresponding temperature information.

[0095] In order to implement the above embodiment, the present invention also proposes an intelligent detection device based on temperature vision.

[0096] Figure 2 A schematic diagram of the structure of a temperature vision-based intelligent detection device provided in an embodiment of the present invention.

[0097] like Figure 2 As shown, the temperature vision-based intelligent detection device 200 may include: a first acquisition module 210 , a second acquisition module 220 , a first determination module 230 , a third acquisition module 240 and a second determination module 250 .

[0098] A first acquisition module is used to acquire a temperature visual image of a target power device;

[0099] A second acquisition module is used to input the temperature visual image into a pre-trained target detection model to obtain type information of the target power equipment;

[0100] A first determination module is used to determine a target area image and a sub-classification model corresponding to the target power equipment according to the type information and location information of the target power equipment, and is specifically used to: crop the temperature visual image of the target power equipment according to the location information of the target power equipment to obtain the target area image, and determine the sub-classification model corresponding to the current target power equipment based on the correspondence between the type information of the target power equipment and the sub-classification model;

[0101] A third acquisition module is used to input the target area image into the sub-classification model to obtain an image to be detected containing each component frame marked in the target area image;

[0102] The second determination module is used to input the image to be detected into a pre-trained fault detection model to determine the fault type and fault location of the target power equipment.

[0103] Optionally, the second acquisition module is further used to:

[0104] Acquire a pre-trained data set of a target power device, wherein the pre-trained data set is a temperature visual data set of the target power device;

[0105] Annotating the temperature visual data set in a specified manner, wherein the specified manner at least includes device frame annotation and feature point annotation;

[0106] The labeled temperature vision dataset is used to train the initial detection model to obtain the target detection model.

[0107] Optionally, the first determining module is further used to:

[0108] Crop each temperature visual image in the pre-training data set of the target power equipment to obtain a training data set containing the labeled area;

[0109] Classifying the training data set according to the type of target power equipment to obtain multiple training data subsets;

[0110] The multiple training data subsets are used to train multiple corresponding initial sub-classification models to obtain multiple trained sub-classification models.

[0111] Optionally, the second determining module is further used to:

[0112] Obtain a binary temperature data set divided into faulty devices and non-faulty devices;

[0113] The initial fault diagnosis model is trained using the binary temperature data set to generate a fault detection model.

[0114] Optionally, the device further comprises:

[0115] A third acquisition module is used to input the image to be detected into the fault detection model generated by training to obtain a label corresponding to the image to be detected, wherein the label includes a fault type label and a fault location label;

[0116] A fourth acquisition module, used for inputting the image to be detected into the initial fault diagnosis model to be trained to obtain a prediction label corresponding to the image to be detected;

[0117] The correction module is used to correct the initial fault diagnosis model and the fault detection model respectively according to the difference between the predicted label and the marked label.

[0118] In an embodiment of the present invention, a temperature visual image of a target power device is first obtained, and then the temperature visual image is input into a pre-trained target detection model to obtain type information of the target power device. Then, based on the type information and location information of the target power device, a target area image and a sub-classification model corresponding to the target power device are determined. Then, the target area image is input into the sub-classification model to obtain an image to be detected that contains each component box marked in the target area image. Finally, the image to be detected is input into a pre-trained fault detection model to determine the fault type and fault location of the target power device. Thus, the temperature visual data can be used to train the target detection model to obtain the type and location of the device in the temperature visual image, the component location of the power device in the image can be locked more accurately, and the fault of the power device can be intelligently diagnosed.

[0119] Based on temperature visual information, the position of power equipment is located through the target detection network, and the position of power equipment components is located through the feature point network, so as to accurately locate the position of each component of the power equipment and obtain the corresponding temperature information.

[0120] In order to implement the above embodiments, the present invention also proposes an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the intelligent detection method based on temperature vision proposed in the above embodiments of the present invention is implemented.

[0121] In order to implement the above embodiments, the present invention further proposes a non-temporary computer-readable storage medium storing a computer program, which, when executed by a processor, implements the intelligent detection method based on temperature vision as proposed in the above embodiments of the present invention.

[0122] In order to implement the above embodiments, the present invention further proposes a computer program product. When the instruction processor in the computer program product is executed, the intelligent detection method based on temperature vision proposed in the above embodiments of the present invention is executed.

[0123] Figure 3 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present invention is shown. Figure 3The electronic device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0124] like Figure 3 As shown, the electronic device 12 is in the form of a general purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0125] The bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of a variety of bus structures. For example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnection (PCI) bus.

[0126] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0127] The memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 3 not shown, usually called a "hard drive"). Although Figure 3Not shown in the figure, a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a compact disk read only memory (hereinafter referred to as: CD-ROM), a digital versatile disk read only memory (hereinafter referred to as: DVD-ROM) or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules, which are configured to perform the functions of the various embodiments of the present invention.

[0128] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0129] The electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 12, and / or communicate with any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 22. In addition, the electronic device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 through a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0130] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the methods mentioned in the above embodiments.

[0131] In an embodiment of the present invention, a temperature visual image of a target power device is first obtained, and then the temperature visual image is input into a pre-trained target detection model to obtain the type information of the target power device. Then, according to the type information and location information of the target power device, a target area image and a sub-classification model corresponding to the target power device are determined, and then the target area image is input into the sub-classification model to obtain an image to be detected containing each component box marked in the target area image. Finally, the image to be detected is input into a pre-trained fault detection model to determine the fault type and fault location of the target power device. Thus, the target detection model can be trained using temperature visual data to obtain the type and location of the device in the temperature visual image, the component position of the power device in the image can be locked more accurately, and the fault of the power device can be intelligently diagnosed. Based on the temperature visual information, the position of the power device is located through the target detection network, and the position of the power device components is located through the feature point network, so as to accurately locate the positions of the components of the power device and obtain the corresponding temperature information.

[0132] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0133] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0134] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0135] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0136] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0137] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0138] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0139] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. An intelligent detection method based on temperature vision, characterized in that: include: Acquire a visual image of the temperature of the target power equipment; Inputting the temperature visual image into a pre-trained target detection model to obtain type information of the target power equipment; Determine a target area image and a sub-classification model corresponding to the target power equipment according to the type information and location information of the target power equipment, including: cropping the temperature visual image of the target power equipment according to the location information of the target power equipment to obtain the target area image, and determining the sub-classification model corresponding to the current target power equipment based on the correspondence between the type information of the target power equipment and the sub-classification model; Inputting the target area image into the sub-classification model to obtain an image to be detected containing each component frame marked in the target area image; The image to be detected is input into a pre-trained fault detection model to determine the fault type and fault location of the target power equipment.

2. The method according to claim 1, characterized in that Before inputting the training data set into the pre-trained target detection model to obtain the type information and location information of the target power equipment, the following steps are also included: Acquire a pre-trained data set of a target power device, wherein the pre-trained data set is a temperature visual data set of the target power device; Annotating the temperature visual data set in a specified manner, wherein the specified manner at least includes device frame annotation and feature point annotation; The labeled temperature vision dataset is used to train the initial detection model to obtain the target detection model.

3. The method according to claim 1, characterized in that Before determining the target area image and the sub-classification model corresponding to the target power equipment according to the type information and location information of the target power equipment, the method further includes: Crop each temperature visual image in the pre-training data set of the target power equipment to obtain a training data set containing the labeled area; Classifying the training data set according to the type of target power equipment to obtain multiple training data subsets; The multiple training data subsets are used to train multiple corresponding initial sub-classification models to obtain multiple trained sub-classification models.

4. The method according to claim 1, characterized in that: Before inputting the image to be detected into the pre-trained fault detection model, the method further includes: Obtain a binary temperature data set divided into faulty devices and non-faulty devices; The initial fault diagnosis model is trained using the binary temperature data set to generate a fault detection model.

5. The method according to claim 1, characterized in that Also includes: Inputting the image to be detected into the fault detection model generated by training to obtain the annotation label corresponding to the image to be detected, wherein the annotation label includes a fault type label and a fault location label; Inputting the image to be detected into the initial fault diagnosis model to be trained to obtain a prediction label corresponding to the image to be detected; According to the difference between the predicted label and the marked label, the initial fault diagnosis model and the fault detection model are respectively modified.

6. An intelligent detection device based on temperature vision, characterized in that: include: A first acquisition module is used to acquire a temperature visual image of a target power device; A second acquisition module is used to input the temperature visual image into a pre-trained target detection model to obtain type information of the target power equipment; A first determination module is used to determine a target area image and a sub-classification model corresponding to the target power equipment according to the type information and location information of the target power equipment, and is specifically used to: crop the temperature visual image of the target power equipment according to the location information of the target power equipment to obtain the target area image, and determine the sub-classification model corresponding to the current target power equipment based on the correspondence between the type information of the target power equipment and the sub-classification model; A third acquisition module is used to input the target area image into the sub-classification model to obtain an image to be detected containing each component frame marked in the target area image; The second determination module is used to input the image to be detected into a pre-trained fault detection model to determine the fault type and fault location of the target power equipment.

7. The device according to claim 6, characterized in that The second acquisition module is further used for: Acquire a pre-trained data set of a target power device, wherein the pre-trained data set is a temperature visual data set of the target power device; Annotating the temperature visual data set in a specified manner, wherein the specified manner at least includes device frame annotation and feature point annotation; The labeled temperature vision dataset is used to train the initial detection model to obtain the target detection model.

8. The device according to claim 7, characterized in that The first determining module is further used for: Crop each temperature visual image in the pre-training data set of the target power equipment to obtain a training data set containing the labeled area; Classifying the training data set according to the type of target power equipment to obtain multiple training data subsets; The multiple training data subsets are used to train multiple corresponding initial sub-classification models to obtain multiple trained sub-classification models.

Images