Power transmission equipment heating defect identification method, electronic equipment and storage medium

Abstract

The invention discloses a power transmission equipment heating defect identification method, electronic equipment and a storage medium, and relates to the technical field of power equipment fault diagnosis. The specific implementation scheme is as follows: acquiring an infrared image obtained by inspecting the power transmission equipment through the unmanned aerial vehicle; generating image quality evaluation based on the image content of the infrared image and the temperature data; and generating a heating defect identification result of the power transmission equipment according to the infrared image of which the image quality evaluation meets a preset condition. According to the embodiment of the invention, the defects of incomplete image quality evaluation and low manual identification efficiency in the prior art are overcome, and the problem that the power transmission equipment area cannot be accurately positioned in the infrared image in the prior art is solved, so that the missing report rate and the false report rate of heating defect identification of the power transmission equipment are reduced; the problem that the heating defect recognition accuracy of the power transmission equipment is not high is solved, and the heating defect recognition efficiency of the power transmission equipment is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of fault diagnosis of electric power equipment, and in particular to a method for identifying heating defects of power transmission equipment, an electronic device and a storage medium. Background Art

[0002] The transmission system is an important part of the power system. During the transmission process, the safety and stability of the transmission equipment are of vital importance. Long-term high-load operation often causes the transmission equipment to heat up, which not only affects the equipment efficiency, but may also cause serious safety accidents. At present, the method of identifying heating defects of transmission equipment is mainly based on the processing and analysis of infrared images supplemented by the judgment of professionals. However, this method is highly dependent on the staff and is prone to misjudgment or missed judgment. In addition, due to the influence of the shooting environment, there are often problems such as low signal-to-noise ratio and contrast of infrared images, large background interference of infrared images, and inability to accurately locate the transmission equipment area. In addition, infrared temperature measurement has strict requirements on the environment and shooting methods, resulting in uneven quality of the acquired infrared images, which affects the efficiency of identifying heating defects of transmission equipment. Summary of the invention

[0003] The present invention provides a method for identifying heating defects of power transmission equipment, an electronic device and a storage medium, which are used to solve the problem that the power transmission equipment area cannot be accurately located in an infrared image and the quality of infrared images is uneven, so that the missed alarm rate and false alarm rate of heating defect identification of the power transmission equipment are reduced, and the beneficial effect of improving the efficiency of heating defect identification is achieved.

[0004] According to one aspect of an embodiment of the present invention, a method for identifying heating defects of power transmission equipment is provided, comprising: acquiring infrared images obtained by inspecting power transmission equipment through a drone; generating an image quality evaluation based on the image content and temperature data of the infrared image; and generating a heating defect identification result of the power transmission equipment based on the infrared image that meets preset conditions according to the image quality evaluation.

[0005] Another aspect of an embodiment of the present invention provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for identifying heating defects of power transmission equipment described in any one of the embodiments of the present invention.

[0006] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions, and the computer instructions are used to enable a processor to implement the method for identifying heating defects of power transmission equipment according to any embodiment of the present invention when executed.

[0007] In the embodiment of the present invention, an infrared image obtained by a drone inspecting a power transmission device is obtained, the image content of the infrared image is detected from the acquired infrared image, and the temperature data of the infrared image is extracted, and the infrared image is analyzed and evaluated based on the temperature data and the image content, respectively, and the result of the analysis and evaluation is used as an image quality evaluation, and the generated quality evaluation is screened according to a preset condition, and the infrared image corresponding to the preset condition is identified for heating defects, and the infrared image is used to determine whether the power transmission device has a heating defect and output the heating defect identification result. In the embodiment of the present invention, the heating defect of the power transmission device is identified based on the image content and temperature data of the infrared image, which overcomes the defects of the prior art in that the image quality evaluation is not comprehensive and the manual recognition rate is low, so that the missed alarm rate and the false alarm rate of the heating defect identification of the power transmission device are reduced, and the infrared image corresponding to the image quality evaluation that meets the preset conditions is used for subsequent heating defect identification of the power transmission device, which solves the problem that the prior art cannot accurately locate the power transmission device area in the infrared image, so that the accuracy rate of the heating defect identification of the power transmission device is improved, and the beneficial effect of improving the efficiency of the heating defect identification of the power transmission device is achieved.

[0008] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 The present invention provides a flow chart of a method for identifying heating defects in power transmission equipment according to an embodiment of the present invention.

[0011] Figure 2 It is a flow chart of another method for identifying heating defects of power transmission equipment provided according to an embodiment of the present invention.

[0012] Figure 3 It is a flow chart of another method for identifying heating defects of power transmission equipment provided according to an embodiment of the present invention.

[0013] Figure 4 It is a flow chart of another method for identifying heating defects of power transmission equipment provided according to an embodiment of the present invention.

[0014] Figure 5 It is a flow chart of another method for identifying heating defects of power transmission equipment provided according to an embodiment of the present invention.

[0015] Figure 6 This is a general view of the composite insulator provided by an embodiment of the present invention.

[0016] Figure 7 This is a diagram of an unshielded core rod of a composite insulator provided in an embodiment of the present invention.

[0017] Figure 8 This is a diagram of a porcelain insulator without a steel cap covered by an embodiment of the present invention.

[0018] Fig. 9 This is a general view of the porcelain insulator provided by an embodiment of the present invention.

[0019] Fig.10 It is a flow chart of another method for identifying heating defects of power transmission equipment provided according to an embodiment of the present invention.

[0020] Fig.11 This is a framework diagram of an improved mask region convolutional neural network provided in an embodiment of the present invention.

[0021] Fig.12 The present invention provides an overall flow chart of heating defect identification of power transmission equipment during drone inspection according to an embodiment of the present invention.

[0022] Fig.13 The present invention provides an infrared image quality evaluation flow chart according to an embodiment of the present invention.

[0023] Fig.14 It is a flow chart of another method for identifying heating defects of power transmission equipment provided according to an embodiment of the present invention.

[0024] Fig.15 This is a general view of the splitting of the crimping hardware provided by the embodiment of the present invention.

[0025] Fig.16 It is a schematic diagram of the structure of a device for implementing a method for identifying heating defects of power transmission equipment provided by an embodiment of the present invention.

[0026] Fig.17 It is a structural schematic diagram of an electronic device for implementing the method for identifying heating defects of power transmission equipment according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] Figure 1 A flowchart of a method for identifying heating defects of power transmission equipment is provided for an embodiment of the present invention. This embodiment is applicable to the case where heating defects of power transmission equipment are identified based on infrared images. The method can be executed by a device for identifying heating defects of power transmission equipment. The device for identifying heating defects of power transmission equipment can be implemented in the form of hardware and / or software. The device for identifying heating defects of power transmission equipment can be configured in an electronic device. Figure 1 As shown, the method includes.

[0030] S110, acquiring infrared images obtained by inspecting power transmission equipment through a drone.

[0031] In the embodiment of the present invention, the power transmission equipment can be understood as various facilities used to transmit electric energy in the power system, wherein the power transmission equipment includes but is not limited to: composite insulators, porcelain insulators and crimping hardware.

[0032] An infrared image is an image showing the temperature distribution on the surface of an object. The infrared image is usually obtained by an infrared imaging device, wherein the infrared imaging device may include an infrared thermal imager or an infrared camera.

[0033] Specifically, an infrared image is obtained from a drone, which is obtained when the drone inspects power transmission equipment. The method of obtaining the infrared image may include: using wireless communication technology to transmit the infrared image collected by the drone in real time or exporting the infrared image stored in the drone by connecting to the drone through an interface after the drone completes the task and returns.

[0034] S120 : Generate an image quality evaluation based on the image content and temperature data of the infrared image.

[0035] In the embodiment of the present invention, the image content can be understood as the visual information of the photographed object presented by the image, wherein the image content includes but is not limited to: the shape, size and background of the photographed object.

[0036] Image quality evaluation can be understood as the result of evaluating the image content or temperature data of the infrared image. Image quality evaluation is used to determine whether the infrared image is suitable for the next step of heating defect identification. For example, image quality evaluation can include: image qualified or image unqualified.

[0037] Specifically, the image content is detected and the temperature data is extracted from the acquired infrared image, the quality of the infrared image is evaluated from two aspects of image content and temperature data, and an image quality evaluation is generated based on the detection results of the image content and the extraction results of the temperature data, wherein two independent image quality evaluations can be generated based on the results of the two aspects or one image quality evaluation can be generated by comprehensively considering the results of the two aspects, and the presentation form of the image quality evaluation can include: chart display or text description.

[0038] S130, generating a heating defect recognition result of the power transmission equipment according to the infrared image that meets the preset conditions according to the image quality evaluation.

[0039] In an embodiment of the present invention, the preset condition can be understood as the minimum evaluation standard for infrared images to identify heating defects, which is used to determine whether the infrared image is suitable for subsequent transmission equipment heating defect identification tasks, wherein the preset condition at least includes: the image quality evaluation generated based on the temperature data of the infrared image and the image quality evaluation generated based on the image content are both qualified images.

[0040] Thermal defect recognition can be understood as the process of determining whether there is abnormal temperature in the power transmission equipment by analyzing infrared images. The thermal defect recognition result is the result obtained by thermal recognition of the power transmission equipment. The thermal defect recognition result may include: the specific location, size and degree of temperature abnormality of the abnormal heating area of ​​the target object such as the power transmission equipment, providing key information for maintenance and fault prevention.

[0041] Specifically, the generated image quality evaluation is compared with the preset conditions, the image quality evaluation that meets the preset conditions is found, and the infrared image corresponding to the image quality evaluation that meets the preset conditions is used to identify heating defects. For example, it is possible to determine whether there is a heating defect in the power transmission equipment by judging whether there is abnormal data in the infrared image or whether the temperature distribution is uniform, and output the heating defect identification result.

[0042] In the embodiment of the present invention, an infrared image obtained by a drone inspecting a power transmission device is obtained, the image content of the infrared image is detected from the acquired infrared image, and the temperature data of the infrared image is extracted, and the infrared image is analyzed and evaluated based on the temperature data and the image content, respectively, and the result of the analysis and evaluation is used as an image quality evaluation, and the generated quality evaluation is screened according to a preset condition, and the infrared image corresponding to the preset condition is identified for heating defects, and the infrared image is used to determine whether the power transmission device has a heating defect and output the heating defect identification result. In the embodiment of the present invention, the heating defect of the power transmission device is identified based on the image content and temperature data of the infrared image, which overcomes the defects of the prior art in that the image quality evaluation is not comprehensive and the manual recognition rate is low, so that the missed alarm rate and the false alarm rate of the heating defect identification of the power transmission device are reduced, and the infrared image corresponding to the image quality evaluation that meets the preset conditions is used for subsequent heating defect identification of the power transmission device, which solves the problem that the prior art cannot accurately locate the power transmission device area in the infrared image, so that the accuracy rate of the heating defect identification of the power transmission device is improved, and the beneficial effect of improving the efficiency of the heating defect identification of the power transmission device is achieved.

[0043] Figure 2 This is a flow chart of a method for identifying heating defects of power transmission equipment provided by an embodiment of the present invention. Based on the above-mentioned embodiment, the embodiment of the present invention provides another method for identifying heating defects of power transmission equipment. Figure 2 As shown, the method includes.

[0044] S210, acquiring infrared images obtained by inspecting power transmission equipment through a drone.

[0045] S220: Call a preset temperature analysis tool to extract temperature data of the infrared image, and generate a first image quality evaluation of the infrared image according to a reading result of the temperature data.

[0046] In the embodiment of the present invention, the temperature analysis tool may be understood as a software for extracting, analyzing and processing temperature data from infrared images. For example, the temperature analysis tool may include Matlab or Testo Thermography Software.

[0047] The reading result may be understood as the result of whether the temperature data can be successfully extracted from the infrared image. For example, the reading result may include: successful extraction or unsuccessful extraction.

[0048] The first image quality evaluation may be understood as a result obtained after evaluation based on the temperature data of the infrared image. For example, the first image quality evaluation may include: the infrared image is valid or the infrared image is invalid.

[0049] Specifically, a pre-configured temperature analysis tool is obtained, and an infrared image is input into the temperature analysis tool. The temperature analysis tool is used to determine whether temperature data can be successfully extracted from the infrared image. The result of whether the temperature data can be successfully extracted is used as a reading result of the temperature data. A first image quality evaluation of the infrared image is generated based on the reading result. If the reading result is that the extraction is successful, the generated first image quality evaluation is valid; if the reading result is that the extraction is unsuccessful, the generated first image quality evaluation is invalid. The electronic device may call the temperature analysis tool in the following ways: direct integrated call or external connection call. Direct integrated call means that the electronic device has a built-in temperature analysis tool, and the temperature analysis tool can be called directly through the operating interface or software platform of the electronic device. External connection call means that the temperature analysis tool is connected to the electronic device through a connection interface, and the electronic device calls the temperature analysis tool to analyze the temperature data by reading the data transmitted by the interface.

[0050] S230 , calling a pre-trained image content analysis model to evaluate the image content of the infrared image, and obtaining a second image quality evaluation of the infrared image.

[0051] In an embodiment of the present invention, the image content analysis model can be understood as a tool for automatically analyzing and understanding the visual information of the photographed object presented in the image, wherein the image content analysis model includes at least one of the following: a support vector machine model, a target detection model, and an image segmentation algorithm model.

[0052] A pre-trained image content analysis model refers to an image content analysis model that has completed the training of infrared images based on a large number of infrared images according to actual needs or standards, and does not require additional settings or adjustments, and can be directly used to analyze and process the image content of infrared images.

[0053] The second image quality evaluation may be understood as a result obtained based on an evaluation of visual information of the photographed object presented in the infrared image. For example, the second image quality evaluation may include: the image is qualified or the image is unqualified.

[0054] Specifically, image content is extracted from the infrared image, the image content is input into an image content analysis model, the image content analysis model is called based on the image content to evaluate the quality of the infrared image and an evaluation result is output, and a second image quality evaluation of the infrared image is generated based on the evaluation result, wherein the image content may include: blur parameters of the infrared image or image background conditions.

[0055] For example, the blur parameter of the infrared image is input as the image content into the image content analysis model, and the image content analysis model evaluates the quality of the infrared image based on the blur parameter and the clarity of the infrared image. If the evaluation result is that the image is clear, the second image quality evaluation of the infrared image is qualified. Or the image background of the infrared image is input as the image content into the image content analysis model, and the image content analysis model evaluates the quality of the infrared image based on whether the image background includes the sky. If the evaluation result is that it includes the sky, the second image quality evaluation of the infrared image is qualified.

[0056] S240, generating a heating defect recognition result of the power transmission equipment according to the infrared image that meets the preset conditions according to the image quality evaluation.

[0057] In an embodiment of the present invention, an infrared image obtained by a drone inspecting a power transmission device is obtained, a preset temperature analysis tool is called, whether temperature data can be extracted from the infrared image is judged and a judgment result is output, the judgment result is used as a reading result of the temperature data, a first image quality evaluation of the infrared image is generated according to the reading result, image content is extracted from the infrared image, the image content is input into an image content analysis model, the image content analysis model is called based on the image content to evaluate the quality of the infrared image and an evaluation result is output, a second image quality evaluation of the infrared image is generated based on the evaluation result, the generated first image quality evaluation and the second image quality evaluation are compared with preset conditions respectively, the first image quality evaluation and the second image quality evaluation that meet the preset conditions are found, the infrared image corresponding to the image quality evaluation that meets the preset conditions is identified for heating defects, and the heating defect identification result is output. In an embodiment of the present invention, the first image quality evaluation focuses on temperature information, and by judging the temperature information of the infrared image, the validity of the temperature information of the image entering the power transmission device heating defect identification image is ensured; the second image quality evaluation focuses on evaluating the image content, and by evaluating the image content of the infrared image, the image entering the power transmission device heating defect identification image is ensured to visually meet the quality requirements. The embodiment of the present invention can comprehensively and accurately evaluate the quality of infrared images through evaluation of these two dimensions, overcoming the defects of the existing technology of incomplete image quality evaluation and low manual recognition rate, thereby reducing the missed alarm rate and false alarm rate.

[0058] On the basis of the above embodiment, a pre-trained image content analysis model is called to evaluate the image content of the infrared image to obtain a second image quality evaluation of the infrared image, including: extracting fuzzy parameters of the infrared image as image content, calling the image content analysis model to classify the fuzzy parameters, and obtaining an image fuzzy classification result as a second image quality evaluation; calling the image content analysis model to detect the specific situation of the image background including the sky in the infrared image as image content, and obtaining a second image quality evaluation of the infrared image according to the image content; calling the image content analysis model to detect the occlusion of the power transmission equipment in the infrared image as image content, and obtaining a second image quality evaluation of the infrared image according to the image content.

[0059] In the embodiment of the present invention, the blur parameter can be understood as a quantitative index reflecting the blur of the infrared image, which is used to objectively reflect the blur status of the image. For example, the blur parameter includes but is not limited to: grayscale variance, Sobel gradient and Roberts gradient.

[0060] Specifically, a quantitative index that can reflect the blurriness of the infrared image is extracted from the infrared image, and the quantitative index is input into the image content analysis model as a blur parameter. Based on the blur parameter, the image content analysis model is used to classify whether the infrared image is clear to obtain an image blur classification result. A second image quality evaluation of the infrared image is generated based on the image blur classification result. If the image blur classification result is that the image is clear, the second image quality evaluation of the generated infrared image is a qualified image. The infrared image is input into the content analysis model, and the image content analysis model is used to detect whether the background of the infrared image includes the sky, and a specific result of whether the background of the infrared image includes the sky is obtained as the image content. Based on the specific result, that is, the image content, a second image quality evaluation of the infrared image is generated. If the specific result is that the sky is included, the second image quality evaluation of the generated infrared image is a qualified image. The infrared image is input into the content analysis model, and the content analysis model is used to detect the occlusion degree of the power transmission equipment in the infrared image, and a specific detection result is output. The occlusion degree of the power transmission equipment in the infrared image can be used as the image content. Based on the specific detection result of the corresponding image content, a second image quality evaluation of the infrared image is generated. If the detection result is that the occlusion degree is low, the second image quality evaluation of the generated infrared image is a qualified image. It can be understood that the second image quality evaluation of the infrared image obtained by evaluating the image content of the infrared image using the image content analysis model includes: obtaining a second image quality evaluation based on an image blur parameter, obtaining a second image quality evaluation based on the specific situation of the image background including the sky, and obtaining a second image quality evaluation based on the occlusion situation of the power transmission equipment.

[0061] Figure 3This is a flow chart of another method for identifying heating defects of power transmission equipment provided by an embodiment of the present invention. Based on the above-mentioned embodiment, the embodiment of the present invention provides another method for identifying heating defects of power transmission equipment. Figure 3 As shown, the method includes.

[0062] S310, obtaining infrared images obtained by inspecting power transmission equipment through a drone.

[0063] S320: Call a preset temperature analysis tool to extract temperature data of the infrared image, and generate a first image quality evaluation of the infrared image according to the reading result of the temperature data.

[0064] S330, obtaining a pre-trained support vector machine model as an image content analysis model, wherein the training data set of the support vector machine model includes at least a clear image data set and a blurred image data set.

[0065] In the embodiment of the present invention, the support vector machine model refers to a supervised learning algorithm model used for image classification.

[0066] The training data set can be understood as a collection of a series of image samples, which is used to ensure that the support vector machine model can learn effective features to distinguish different image contents. The training data set includes at least a clear image data set and a blurred image data set, wherein the training data set can be obtained by downloading from a public database or by self-photographing and processing, which is not limited in the embodiment of the present invention.

[0067] Specifically, a pre-trained support vector machine model is obtained, and the pre-trained support vector machine model is obtained by training at least two types of data sets, such as a clear image data set and a blurred image data set. The pre-trained support vector machine model is used as an image content analysis model to determine whether the infrared image is clear. The training data set can be obtained by downloading it from a public database or taking it by yourself and annotating it.

[0068] S340, extracting fuzzy parameters of the infrared image, wherein the fuzzy parameters include at least one of the following: grayscale variance, Sobel gradient, Roberts gradient, Laplace gradient, energy gradient, and fast Fourier transform domain value.

[0069] In the embodiment of the present invention, at least one quantitative index such as grayscale variance, Sobel gradient, Roberts gradient, Laplace gradient, energy gradient and fast Fourier transform domain value can be used as the fuzzy parameter of the infrared image, wherein the method of obtaining the grayscale variance from the infrared image may include: converting the infrared image into a grayscale image, calculating the grayscale histogram of the grayscale image, and calculating the variance of the grayscale value according to the grayscale histogram; the method of obtaining the Sobel gradient from the infrared image may include: using the Sobel operator to perform a convolution operation on the infrared image to obtain two gradient images, and then calculating the gradient amplitude of each pixel point according to the gradient image, that is, calculating the Sobel gradient of the infrared image. Among them, the method of extracting the grayscale variance, Sobel gradient, Roberts gradient, Laplace gradient, energy gradient and fast Fourier transform domain value of the infrared image from the infrared image is a common method in the field of image processing, and the embodiment of the present invention will not be described in detail.

[0070] Specifically, at least one quantitative index that can reflect the blurriness of the infrared image, such as grayscale variance, Sobel gradient, Roberts gradient, Laplace gradient, energy gradient and fast Fourier transform domain value, is extracted from the infrared image, and the quantitative index is used as the blur parameter of the infrared image.

[0071] S350, inputting the fuzzy parameters into the support vector machine model to obtain the image fuzzy classification result of the support vector machine model corresponding to the infrared image.

[0072] In the embodiment of the present invention, the image blur classification result can be understood as the result of classifying whether the infrared image is clear based on the infrared blur parameter. For example, the image blur classification result may include: the image is clear or the image is blurred.

[0073] Specifically, the fuzzy parameters extracted from the infrared image are input into the support vector machine model, and based on the fuzzy parameters, the support vector machine model is called to classify whether the infrared image is clear and the classification result is output, and the classification result is used as the image fuzzy classification result of the infrared image. If the infrared image is clear, then the image fuzzy classification result is that the image is clear; if the infrared image is fuzzy, then the image fuzzy classification result is that the image is fuzzy. The input of the support vector machine model is usually a feature vector composed of fuzzy parameters, and each eigenvalue of the feature vector corresponds to a fuzzy parameter. All the extracted fuzzy parameters can be combined into a feature vector as the input of the support vector machine model, or the extracted fuzzy parameters can be grouped to form multiple feature vectors as the input of the support vector machine model. The embodiment of the present invention does not limit this.

[0074] S360: If the image blur classification result is that the image is clear, a second image with a qualified image quality evaluation is generated; if the image blur classification result is that the image is blurred, a second image with an unqualified image quality evaluation is generated.

[0075] Specifically, the quality evaluation of the second generated image is evaluated again based on the image blur classification result of the infrared image. If the image blur classification result is that the image is clear, then the quality evaluation of the second generated image is a qualified image. If the image blur classification result is that the image is blurred, then the quality evaluation of the second generated image is an unqualified image.

[0076] S370, generating a heating defect recognition result of the power transmission equipment based on the infrared image that meets the preset conditions according to the image quality evaluation.

[0077] In an embodiment of the present invention, an infrared image obtained by inspecting a power transmission device by a drone is obtained, a preset temperature analysis tool is called, whether temperature data can be extracted from the infrared image is judged and a reading result is output, a first image quality evaluation of the infrared image is generated according to the reading result, a pre-trained support vector machine model is used as an image content analysis model, a fuzzy parameter extracted from the infrared image is used as an input of the support vector machine model, and based on the fuzzy parameter, the infrared image is classified to see whether it is clear by calling the support vector machine model to obtain an image fuzzy classification result, and a second image quality evaluation is judged again based on the image fuzzy classification result. If the image fuzzy classification result is that the image is clear, then the generated second image quality evaluation is a qualified image, and if the image fuzzy classification result is that the image is fuzzy, then the generated second image quality evaluation is an unqualified image, and the generated first image quality evaluation and second image quality evaluation are respectively compared with preset conditions to find out the first image quality evaluation and the second image quality evaluation that meet the preset conditions, and the infrared image corresponding to the image quality evaluation that meets the preset conditions is subsequently identified for heating defects, and the identification result is output, and the identified result is used as the heating defect result of the power transmission device. In the embodiment of the present invention, at least one fuzzy parameter such as the grayscale variance, Sobel gradient, Roberts gradient, Laplace gradient, energy gradient and fast Fourier transform domain value of the infrared image is used as the input of the support vector machine model, so that the support vector machine model can comprehensively analyze the quantitative indicators of the infrared image blur, so that the support vector machine model can classify the infrared image blur more accurately and comprehensively, and reduce the omission rate and false alarm rate.

[0078] Figure 4 This is a flow chart of another method for identifying heating defects of power transmission equipment provided by an embodiment of the present invention. Based on the above-mentioned embodiment, the embodiment of the present invention provides another method for identifying heating defects of power transmission equipment. Figure 4 As shown, the method includes.

[0079] S410, acquiring infrared images obtained by inspecting power transmission equipment through a drone.

[0080] S420: Call a preset temperature analysis tool to extract temperature data of the infrared image, and generate a first image quality evaluation of the infrared image according to the reading result of the temperature data.

[0081] S430. Obtain a pre-trained target detection model as an image content analysis model, wherein a training data set of the target detection model includes at least labeled images of power transmission equipment, and information labeled in the training data set includes sky and non-sky.

[0082] In the embodiment of the present invention, the target detection model refers to a model that can automatically identify and locate the target object of interest and give its position and category information. The target detection model can be trained based on different target detection algorithms. For example, the target detection model can be trained using the YOLO series algorithm or using a single-stage detection target detection algorithm. The embodiment of the present invention does not limit the target detection algorithm.

[0083] The power transmission equipment image may be understood as an image including the power transmission equipment. For example, the power transmission equipment image may include an image including a composite insulator or an image including a porcelain insulator.

[0084] Specifically, a pre-trained target detection model is obtained, and the pre-trained target detection model is obtained by training on at least one type of training data set, such as images of power transmission equipment that have been labeled as sky or non-sky. The pre-trained target detection model is used as an image content analysis model to analyze and process the image content, wherein the training data set acquisition method may include: downloading from a public database or taking and labeling by oneself.

[0085] S440, inputting the infrared image into an image content analysis model to obtain a sky recognition result in the infrared image.

[0086] In the embodiment of the present invention, the sky recognition result may be understood as a specific recognition result of the image background of the infrared image. For example, the sky recognition result may include whether the sky is recognized within the image background of the infrared image or whether the sky is not recognized.

[0087] Specifically, the infrared image is input into the image content analysis model, and the image content analysis model is used to identify whether the infrared image includes the sky, determine whether the image background is the sky, and output the recognition result. The recognition result is recorded as the sky recognition result. If the image background of the infrared image is the sky, then the sky recognition result is that the sky is recognized. If the image background of the infrared image is the sky, then the sky recognition result is that the sky is not recognized.

[0088] S450: If the sky recognition result is that the sky is recognized, a second image is generated with a qualified image quality evaluation; if the sky recognition result is that the sky is not recognized, the second image is generated with a failed image quality evaluation.

[0089] Specifically, the quality evaluation of the two generated images is evaluated again based on the sky recognition result. If the sky recognition result is that the sky is recognized, then the quality evaluation of the second generated image is a qualified image. If the sky recognition result is that the sky is not recognized, then the quality evaluation of the second generated image is an unqualified image.

[0090] S460, generating a heating defect recognition result of the power transmission equipment according to the infrared image that meets the preset conditions according to the image quality evaluation.

[0091] In an embodiment of the present invention, an infrared image obtained by a drone inspecting a power transmission device is obtained, a preset temperature analysis tool is called, whether temperature data can be extracted from the infrared image is judged and a reading result is output, a first image quality evaluation of the infrared image is generated according to the reading result, a pre-trained target detection model is used as an image content analysis model, whether the infrared image includes the specific situation of the sky as the image content is input into the target detection model, the image background of the infrared image is identified based on the target detection model, the image background of the infrared image is judged to include the sky in the image background, and a sky recognition result is obtained, and a second image quality evaluation is determined based on the sky recognition result. If the sky recognition result is that the sky is recognized, the generated second image quality evaluation is a qualified image, and if the sky recognition result is that the sky is not recognized, the generated second image quality evaluation is an unqualified image, and the infrared images whose first image quality evaluation and second image quality evaluation meet the preset conditions are screened, and the heating defect is identified based on the infrared image that meets the preset conditions, and the recognition result is output, and the recognized result is used as the heating defect result of the power transmission device. In an embodiment of the present invention, by detecting the specific situation of the sky included in the infrared image, the quality of the infrared image can be more comprehensively evaluated, the problem of uneven quality of the background situation of the infrared image is improved, and the false alarm rate and the false alarm rate are reduced.

[0092] Figure 5 This is a flow chart of another method for identifying heating defects of power transmission equipment provided by an embodiment of the present invention. Based on the above-mentioned embodiment, the embodiment of the present invention provides another method for identifying heating defects of power transmission equipment. Figure 5 As shown, the method includes.

[0093] S510, acquiring infrared images obtained by inspecting power transmission equipment through a drone.

[0094] S520: Call a preset temperature analysis tool to extract temperature data of the infrared image, and generate a first image quality evaluation of the infrared image according to the reading result of the temperature data.

[0095] S530, obtaining an image segmentation algorithm of the image content analysis model, and calling the image segmentation algorithm to extract a segmented image corresponding to the power transmission equipment in the infrared image, wherein the image segmentation algorithm at least includes a mask area convolutional neural network with an attention mechanism added.

[0096] In the embodiment of the present invention, the segmented image can be understood as an image specifically focused on the power transmission equipment area. In this image, the non-power transmission equipment area will be eliminated or significantly faded. For example, the segmented image may include: an image containing only composite insulators or an image containing only porcelain insulators.

[0097] Specifically, an image segmentation algorithm of an image content analysis model is obtained, and the infrared image is segmented using the image segmentation algorithm to effectively distinguish and segment the power transmission equipment in the infrared image from the background, thereby obtaining an image specifically focused on the power transmission equipment area, and using this image as a segmented image of the power transmission equipment, wherein the image segmentation algorithm at least uses a masked region convolutional neural network integrated with an attention mechanism, and integrating the attention mechanism with the masked region convolutional neural network may include: adding an attention mechanism to each layer of the backbone network, applying the attention mechanism before passing the feature map to the Region Proposal Network (RPN), or integrating the attention mechanism in the Head network.

[0098] S540: Obtain an occlusion ratio calculation algorithm of the image content analysis model, and count the occlusion ratios of each segmented image according to the occlusion ratio calculation algorithm.

[0099] In the embodiment of the present invention, the occlusion ratio calculation algorithm can be understood as a method for calculating the ratio of the length of the unoccluded portion of the target object in the segmented image to the overall length of the target object, which is used to evaluate the occlusion condition of the target object.

[0100] The shielding ratio can be understood as the ratio of the unshielded portion of the target object to the overall length of the target object, and is used to evaluate the shielding degree of the unshielded portion of the target object. For example, the ratio of the length of the unshielded core rod of the composite insulator to the length of the entire composite insulator is 1:2 or 2:3, then the shielding ratio may include: 1 / 2 or 2 / 3.

[0101] Specifically, an image content analysis model is obtained, and the segmented image is processed using the image content analysis model to obtain the length of the unobstructed portion of the target object in the segmented image and the overall length of the target object, and an occlusion ratio calculation algorithm in the image content analysis model is called to calculate the ratio of the length of the unobstructed portion of the target object in the segmented image to the overall length of the target object to obtain a specific ratio, and this specific ratio is used as the occlusion ratio of the segmented image, and the occlusion ratio calculation algorithm is applied to multiple segmented images to calculate the occlusion ratio, and the occlusion ratio calculation algorithm is used to count the occlusion ratio of each segmented image.

[0102] S550: If the occlusion ratio is greater than or equal to the threshold ratio, a second image is generated with the quality evaluation as a qualified image; if the occlusion ratio is less than the threshold ratio, the second image is generated with the quality evaluation as an unqualified image.

[0103] In the embodiment of the present invention, the threshold ratio can be understood as a preset demarcation value, and for example, the threshold ratio can include 0.5 or 0.6. The threshold ratio is used to define the ratio of the unobstructed part of the image to the whole image, and the image is regarded as a qualified image, wherein the threshold ratio can be set according to the application scenario and the specific requirements of the user for the image quality.

[0104] Specifically, a predefined threshold ratio is obtained, the occlusion ratio is compared with the threshold ratio, and the quality evaluation of the second image generated based on the occlusion situation of the infrared image is determined based on the comparison result. If the occlusion ratio is greater than or equal to the threshold ratio, the quality evaluation of the generated second image is a qualified image; if the occlusion ratio is less than the threshold ratio, the quality evaluation of the generated second image is an unqualified image.

[0105] S560, generating a heating defect recognition result of the power transmission equipment according to the infrared image that meets the preset conditions according to the image quality evaluation.

[0106] In an embodiment of the present invention, an infrared image obtained by a drone inspecting a power transmission device is obtained, a preset temperature analysis tool is called to determine whether temperature data can be extracted from the infrared image and a reading result is output, a first image quality evaluation of the infrared image is generated according to the reading result, the infrared image is processed by calling an image content analysis model including at least an image segmentation algorithm of a mask region convolutional neural network with an attention mechanism added to obtain the length of an unobstructed portion of a target object in the segmented image and the overall length of the target object, an occlusion ratio calculation algorithm in the image content analysis model is called to calculate the ratio of the length of the unobstructed portion of the target object in the segmented image to the overall length of the target object, and a quality evaluation of the segmented image is obtained. The occlusion ratio is compared with the ratio threshold, and the second image quality evaluation is judged again. If the occlusion ratio reaches or exceeds the set ratio threshold, the generated second image quality evaluation is a qualified image; if the occlusion ratio is lower than the ratio threshold, the generated second image quality evaluation is an unqualified image. The generated first image quality evaluation and second image quality evaluation are compared with the preset conditions respectively, and the first image quality evaluation and the second image quality evaluation that meet the preset conditions are found. The infrared image corresponding to the image quality evaluation that meets the preset conditions is used for subsequent heating defect identification, and the identification result is output, and the identified result is used as the heating defect result of the power transmission equipment.

[0107] In the embodiment of the present invention, an attention mechanism is added to the image content analysis model so that the image content analysis model has enhanced ability to capture key features of power transmission equipment, thereby solving the problems of large background interference and low manual recognition efficiency when identifying heating defects of power transmission equipment based on infrared images, and achieving the beneficial effect of accurately extracting and segmenting specific areas of the power transmission equipment; by comparing the occlusion ratio with the threshold ratio, images with unqualified occlusion ratios are excluded, thereby ensuring the accuracy and reliability of the identification of heating defects of power transmission equipment and reducing the omission rate and false alarm rate.

[0108] On the basis of the above embodiment, the occlusion ratio of each segmented image is counted according to the occlusion ratio calculation algorithm, including at least one of the following: counting the first minimum circumscribed rectangle of the whole composite insulator in the segmented image and the second minimum circumscribed rectangle of the unobstructed core rod, and taking the first ratio of the rectangular long side of the second minimum circumscribed rectangle to the rectangular long side of the first minimum circumscribed rectangle as the occlusion ratio; counting the third minimum circumscribed rectangle of the whole porcelain insulator in the segmented image and the unobstructed length of the porcelain insulator steel cap, and taking the second ratio of the unobstructed length to the rectangular long side of the third minimum circumscribed rectangle as the occlusion ratio; wherein the unobstructed length can be determined by the segmentation boundary box of the porcelain insulator steel cap and the pixel points of the porcelain insulator steel cap.

[0109] In the embodiment of the present invention, the first minimum circumscribed rectangle can be understood as an irregular polygonal frame with the smallest area that can completely contain the entire composite insulator, and is used to describe the overall range of the composite insulator in the image. Figure 6 As shown, the first minimum circumscribed rectangle of the composite insulator is an irregular polygon outlined by dotted solid lines.

[0110] The second minimum circumscribed rectangle can be understood as an irregular polygonal frame with the smallest area that can completely contain the unshielded core rod portion of the composite insulator, and is used to reflect the size of the unshielded core rod of the composite insulator. Figure 7 As shown, the second minimum circumscribed rectangle of the unshielded core rod is an irregular polygon outlined by dotted solid lines.

[0111] The first ratio can be understood as a specific numerical value, which is the ratio of the long side of the second minimum circumscribed rectangle to the long side of the first minimum circumscribed rectangle, wherein the units of the long side of the first minimum circumscribed rectangle and the long side of the second minimum circumscribed rectangle may include the number of pixels or physical units converted according to the actual scale, and the embodiment of the present invention is not limited to this.

[0112] For example, if the long side of the first minimum circumscribed rectangle is 100 pixels and the long side of the second minimum circumscribed rectangle is 80 pixels, then the first ratio is equal to 0.8, indicating that the length of the unshielded core rod of the composite insulator accounts for 80% of the length of the entire composite insulator.

[0113] The unobstructed length can be understood as the total length of the unobstructed part of the porcelain insulator steel cap in the image, which can be determined by the segmentation boundary box of the porcelain insulator steel cap and the pixel points of the porcelain insulator steel cap. Among them, the segmentation boundary box can be understood as the boundary of the unobstructed porcelain insulator steel cap, which is used to define the exact position and shape of the unobstructed porcelain insulator steel cap. Figure 8 As shown, the segmentation boundary box of the porcelain insulator steel cap is an irregular polygon outlined by dotted solid lines.

[0114] The third minimum circumscribed rectangle can be understood as an irregular polygonal box with the smallest area that can completely contain the entire porcelain insulator, and is used to describe the complete shape of the porcelain insulator in the image. Fig. 9 As shown, the third smallest circumscribed rectangle of the entire porcelain insulator is an irregular polygon outlined by dotted solid lines.

[0115] The second ratio can be understood as a specific value, which is the ratio between the long side of the third minimum circumscribed rectangle and the unobstructed length, wherein the unobstructed length can be determined by the segmentation boundary box of the porcelain insulator steel cap and the pixel points of the porcelain insulator steel cap. The long side of the third minimum circumscribed rectangle and the unobstructed length unit may include the number of pixels or a physical unit converted according to an actual scale, which is not limited in the embodiment of the present invention.

[0116] For example, the long side of the third minimum circumscribed rectangle is 100 pixels, and the unobstructed length is 70 pixels. Then, the second ratio is equal to 0.7, indicating that the unobstructed length of the porcelain insulator steel cap accounts for 70% of the entire length of the porcelain insulator.

[0117] Specifically, an image segmentation algorithm is called, and a first minimum circumscribed rectangle that can completely cover the entire composite insulator and a second minimum circumscribed rectangle that completely covers the unobstructed core rod portion of the composite insulator are divided by the image segmentation algorithm. The long sides of the first minimum circumscribed rectangle and the second minimum circumscribed rectangle are compared to obtain a first ratio, and the first ratio is used as the occlusion ratio of the composite insulator segmentation image; a third minimum circumscribed rectangle that can completely cover the entire porcelain insulator is divided by the image segmentation algorithm, and a segmentation boundary box that can completely cover the unobstructed steel cap portion of the porcelain insulator is divided, and the unobstructed length of the porcelain insulator steel cap is determined by the segmentation boundary box and the pixel points of the unobstructed steel cap of the porcelain insulator, and the unobstructed length is compared with the long side length of the third minimum circumscribed rectangle to obtain a second ratio, and the second ratio is used as the occlusion ratio of the porcelain insulator segmentation image.

[0118] Fig.10 This is a flow chart of another method for identifying heating defects of power transmission equipment provided by an embodiment of the present invention. Based on the above-mentioned embodiment, the embodiment of the present invention provides another method for identifying heating defects of power transmission equipment. Fig.10 As shown, the method includes.

[0119] S610: Acquire infrared images obtained by inspecting power transmission equipment through a drone.

[0120] S620: Generate an image quality evaluation based on the image content and temperature data of the infrared image.

[0121] S630: Determine that the first image quality evaluation corresponding to the temperature data in the image quality evaluation and the second image quality evaluation corresponding to the image content are both qualified images, and then determine that the image quality evaluation meets the preset conditions.

[0122] Specifically, the quality of the infrared image is evaluated from two aspects: the temperature data and the image content of the infrared image. A first image quality evaluation of the infrared image is generated based on the reading result of the temperature data, and a second image quality evaluation of the infrared image is generated based on the recognition detection result of the image content. If both the first image quality evaluation and the second image quality evaluation are qualified images, it means that the image quality evaluation generated based on the image content and temperature data of the infrared image meets the preset conditions.

[0123] S640: Extract temperature data of key points corresponding to the image area of ​​the power transmission equipment in the infrared image.

[0124] In the embodiment of the present invention, the image area can be understood as an area corresponding to a specific component or structure on the power transmission equipment divided by an image segmentation algorithm, wherein the image area at least includes a composite insulator core rod area, a porcelain insulator steel cap area and a crimping hardware area.

[0125] The key points may be understood as important positions in the image region that are selected based on actual application requirements and image features. For example, the key points may include: a center position and a bottom position.

[0126] Specifically, an image content analysis model is obtained, and an image segmentation algorithm of the image content analysis model is called. The image segmentation algorithm is used to identify and divide the area of ​​specific components or structures on the power transmission equipment from the infrared image, and this area is used as the image area of ​​the power transmission equipment. Key points are selected in the image area, and temperature values ​​of the selected key points are extracted using a temperature analysis tool. The temperature values ​​of the extracted key points are used as key point temperature data, wherein the number of key points may include: one or more.

[0127] S650, matching defect rules of corresponding power transmission equipment in a preset defect rule library according to key point temperature data, wherein the defect rules include a maximum temperature dividing line value and a relative temperature difference dividing line value of the power transmission equipment.

[0128] In the embodiment of the present invention, the defect rule base can be understood as a database containing multiple power transmission equipment defect determination criteria, which is used to identify power transmission equipment defects based on key point temperature data. For example, the defect rule base can include determination rules for composite insulator heating defects, porcelain insulator heating defects, and crimping hardware heating defects.

[0129] Defect rules can be understood as different specific rules defined for different power transmission equipment in the defect rule library, which are used to guide how to judge whether the power transmission equipment has defects based on temperature data. For example, defect rules may include: when the maximum temperature at the center of the composite insulator core rod area exceeds 80 degrees Celsius, it is judged that the composite insulator may be damaged; or when the maximum temperature at the center of the porcelain insulator steel cap area exceeds 70 degrees Celsius, it is considered that there are cracks in the porcelain insulator steel cap area.

[0130] The maximum temperature demarcation value can be understood as a temperature threshold set in the defect rule, which is used to determine whether the temperature of a key point of the transmission equipment exceeds the maximum temperature allowed for normal operation. When the measured temperature exceeds this value, the equipment is considered to have a heating defect. For example, the defect rule may include: stipulating that the maximum temperature demarcation value of the composite insulator is 60 degrees Celsius or the maximum temperature demarcation value of the porcelain insulator is 65 degrees Celsius, indicating that the maximum temperature of the composite insulator core rod area cannot exceed 60 degrees Celsius or the maximum temperature of the porcelain insulator steel cap area cannot exceed 60 degrees Celsius.

[0131] The maximum relative temperature difference demarcation line value can be understood as the limit of the temperature difference between different key points of the power transmission equipment, which is used to determine whether the temperature of certain key points of the power transmission equipment exceeds the maximum temperature difference allowed for normal operation. When the measured relative temperature difference exceeds this value, it is considered that the equipment has a heating defect. For example, the defect rule may include: stipulating that the maximum relative temperature difference demarcation line value of the composite insulator is 40 degrees Celsius or the relative temperature difference demarcation line value of the porcelain insulator is 45 degrees Celsius, indicating that the difference between the maximum temperature and the minimum temperature in the core rod area of ​​the composite insulator cannot exceed 40 degrees Celsius or the difference between the maximum temperature and the minimum temperature in the steel cap area of ​​the porcelain insulator cannot exceed 45 degrees Celsius. According to the key point temperature data, the defect rules of the corresponding power transmission equipment are matched in the preset defect rule library, wherein the defect rules include the maximum temperature demarcation line value and the relative temperature difference demarcation line value of the power transmission equipment.

[0132] Specifically, there are differences in key point temperature data of different power transmission equipment. According to the extracted key point temperature data, the defect rules of the corresponding power transmission equipment are searched in the preset defect rule library. After the defect rules of the corresponding power transmission equipment are found, specific defect rules are matched from the power transmission equipment defect rules according to the key point temperature data, wherein the defect rules of the power transmission equipment at least include: a rule for judging whether the temperature of a key point of the power transmission equipment exceeds the maximum temperature boundary value allowed for normal operation and a rule for judging whether the temperature of certain key points of the power transmission equipment exceeds the maximum relative temperature difference boundary value allowed for normal operation.

[0133] S660: Use the heating defect associated with the defect rule as a heating defect identification result of the power transmission equipment.

[0134] Specifically, a specific defect rule corresponding to the key point temperature data is matched in the defect rule of the corresponding power transmission equipment, and after matching, the heating defect associated with the specific defect rule is used as the heating defect identification result of the power transmission equipment, wherein the heating defect may include: poor contact or damage to the power transmission equipment.

[0135] In an embodiment of the present invention, an infrared image obtained by a drone inspecting a power transmission device is obtained, image content and temperature data of the infrared image are identified and extracted from the obtained infrared image, and the infrared image is analyzed and evaluated based on the temperature data and the image content, respectively, and the results of the analysis and evaluation are used as image quality evaluation. If a first image quality evaluation generated based on a reading result of the temperature data and a second image quality evaluation generated based on a recognition detection result of the image content are both qualified images, it is determined that the image quality evaluation meets a preset condition, and an image segmentation algorithm is used to segment a specific area where the power transmission device is located in the infrared image corresponding to the image quality evaluation that meets the preset condition, and the obtained segmented area is used as the image area of ​​the power transmission device, one or more key points are selected in the image area, and a temperature analysis tool is used to extract the temperature value of the selected key point from the image area, and the extracted temperature value of the key point is used as the key point temperature data, and a specific defect rule corresponding to the key point temperature data is matched in the defect rule corresponding to the power transmission device, and after matching, the heating defect associated with the specific defect rule is used as the heating defect recognition result of the power transmission device.

[0136] In the embodiment of the present invention, key points are selected from the image area, so that the time for extracting temperature data from the infrared image is reduced, thereby achieving the beneficial effect of improving the efficiency of heating defect recognition; the highest temperature and the lowest temperature of the segmented area are matched with the defect rule, so that it can be accurately judged whether the temperature of the power transmission equipment exceeds the normal operating range, thereby improving the accuracy of heating defect recognition of the power transmission equipment; adding an attention mechanism enhances the ability of the image content analysis model to capture key features of the power transmission equipment, achieving the beneficial effect of effectively extracting and segmenting specific areas of the power transmission equipment, thereby improving the efficiency of heating defect recognition of the power transmission equipment.

[0137] On the basis of the above-mentioned embodiment, the key point temperature data corresponding to the image area of ​​the power transmission equipment is extracted from the infrared image, including: obtaining a pre-trained image segmentation algorithm, wherein the image segmentation algorithm at least includes a mask area convolutional neural network with an attention mechanism added; calling the image segmentation algorithm to segment the infrared image to obtain the image area of ​​the power transmission equipment, wherein the image area at least includes a composite insulator core rod area, a porcelain insulator steel cap area, and a crimping hardware area; and respectively extracting the temperature data of at least one preset key point in each image area as the key point temperature data.

[0138] Specifically, a pre-trained image segmentation algorithm is obtained, and the infrared image is segmented using the image segmentation algorithm, wherein the image segmentation algorithm at least uses a masked area convolutional neural network that integrates an attention mechanism to obtain an area of ​​a specific component or structure on the power transmission equipment, wherein this area at least includes; a composite insulator core rod area, a porcelain insulator steel cap area, and a crimping hardware area, and the segmented area of ​​the specific component or structure on the power transmission equipment is used as the image area of ​​the power transmission equipment, at least one key point is selected in the image area, and a temperature analysis tool is used to extract the temperature value of the selected key point, and the extracted temperature value of the key point is used as the key point temperature data.

[0139] On the basis of the above embodiments, the masked region convolutional neural network with an added attention mechanism includes at least: a backbone network, a region proposal network, a region of interest pooling layer, and a Head network, wherein the backbone network, the region proposal network, and the Head network respectively add convolutional attention modules.

[0140] In an embodiment of the present invention, a backbone network refers to a network used to extract features from an input infrared image. For example, the backbone network may include: a residual network or a visual geometry group network; a region proposal network refers to a network used to slide a window on a feature map extracted by the backbone network and generate a region proposal through a convolution operation; a region of interest pooling layer refers to a structure used to convert a feature map region corresponding to each region proposal into a feature vector of a fixed size; a head network refers to a network used to detect and segment a target based on an input feature vector, wherein a region proposal network (RPN) structure includes: a 3×3 convolutional (Conv) layer, a classification branch, and a regression branch. The 3×3 Conv layer further extracts features from the input feature map to generate a new feature map. The classification branch uses a 1×1 Conv layer to generate a predicted value for the classification task, and calculates the probability of each candidate region belonging to the foreground or background through a Softmax function. The regression branch uses another 1×1 The Conv layer generates the regression parameters of the bounding box. In an embodiment of the present invention, the regression parameters of the bounding box are defined as bounding box regression. The Head network includes: an instance segmentation branch and a target detection and classification branch. When performing a target detection task, the target detection and classification branch is responsible for receiving the feature vector output by the interest region pooling layer, and performing target detection and classification on each region proposal to determine whether it contains a target and the category of the target. When performing an image segmentation task, the instance segmentation branch generates a binary mask for each target, which is used to accurately segment the pixel area of ​​the target.

[0141] The convolutional attention module is an attention mechanism for convolutional neural networks. The convolutional attention module can improve the performance of convolutional neural networks through two dimensions: channel and space.

[0142] Specifically, the architecture of the masked region convolutional neural network includes at least four parts: backbone network, region proposal network, interest region pooling layer and head network. The backbone network extracts the features of the infrared image, and the output feature map will be used as the input of the region proposal network. The region proposal network slides a window on the feature map and generates a region proposal. For each region proposal, the interest region pooling layer is responsible for converting its corresponding feature map area into a feature vector of a fixed size. The feature vector output by the interest region pooling layer is input into the head network to achieve accurate detection and segmentation of targets in infrared images. The convolutional attention module is added to the original backbone network, region proposal network and head network of the masked region convolutional neural network, such as Fig.11 As shown in the figure, adding a convolutional attention module to the masked region convolutional neural network can improve the attention of the masked region convolutional neural network to the key areas in the input infrared image, thereby improving the accuracy and efficiency of target detection and instance segmentation.

[0143] On the basis of the above-mentioned embodiments, the process of constructing a preset defect rule library includes: collecting normal infrared image samples and heating defect infrared image samples of different types of power transmission equipment; extracting temperature data of the image key equipment area corresponding to the power transmission equipment in the normal infrared image samples and the heating defect infrared image samples, wherein the temperature data includes at least one of the following: the highest temperature, the lowest temperature, the average temperature and the relative temperature difference; for different types of power transmission equipment, comparing the temperature data of each image key equipment area of ​​the normal infrared image samples and the heating defect infrared image samples, and obtaining the highest temperature and the relative temperature difference of the corresponding image key equipment area. The image key equipment area includes at least one of the following: a composite insulator core rod area, a porcelain insulator steel cap area and a crimping hardware area; storing the image key equipment area of ​​each power transmission equipment and the boundary value of the image key equipment area as the defect rules of the preset defect rule library.

[0144] In the embodiment of the present invention, the key equipment area of ​​the image can be understood as the area corresponding to the specific components of the power transmission equipment in the infrared image. For example, the specific components of the power transmission equipment may include: composite insulator core rods, porcelain insulator steel caps or crimping hardware, etc.

[0145] Specifically, in order to construct preset defect rules for different types of power transmission equipment, it is necessary to collect normal infrared image samples and heating defect infrared image samples corresponding to different types of power transmission equipment, and extract temperature data of key equipment areas in normal infrared image samples and heating defect infrared image samples according to the type of power transmission equipment, wherein at least the highest temperature, lowest temperature, average temperature and relative temperature difference of the key equipment area of ​​the image are extracted. For different types of power transmission equipment, it is necessary to compare the temperature data extracted from normal infrared image samples and heating defect infrared image samples according to the type, and obtain the temperature data of the key equipment area of ​​the corresponding power transmission equipment image. The maximum temperature dividing line value and the relative temperature difference dividing line value store the dividing line values ​​of different types of power transmission equipment and their corresponding image key equipment areas as defect rules of a preset defect rule library, wherein the defect rules at least include the composite insulator core rod area and the dividing line value corresponding to the composite insulator core rod area, the porcelain insulator steel cap area and the dividing line value corresponding to the porcelain insulator steel cap area, or the crimping hardware area and the dividing line value corresponding to the crimping hardware area. The dividing line values ​​at least include: the maximum temperature dividing line value and the relative temperature difference dividing line value. The defect rules of different types of power transmission equipment together constitute a preset defect rule library.

[0146] Based on the above embodiments, the embodiments of the present invention provide an overall process for identifying heating defects of power transmission equipment during drone inspection, an infrared image quality evaluation process, and another method process for identifying heating defects of power transmission equipment.

[0147] like Fig.12As shown in the figure, the process of identifying heating defects of power transmission equipment is described in detail. UAVs perform automatic inspections on power transmission equipment, wherein the inspection points include at least one of the following: insulators and crimping fittings. The infrared images taken by the drones are transmitted back to the intelligent recognition server. The quality of the infrared images inspected by the drones is evaluated, and images that affect the defect judgment criteria due to shooting occlusion, shooting background, and shooting position are excluded. Corresponding image quality evaluations are generated for shooting occlusion, shooting background, and shooting position, respectively. Images with qualified image quality evaluations are input into the improved Mask Region-based Convolutional Neural Network (Mask RCNN). Based on the improved Mask The RCNN algorithm performs pixel-level positioning of the power transmission equipment in the infrared image, obtains a segmented image of the power transmission equipment, performs temperature analysis on the segmented image, and performs defect diagnosis. If there is no defect, the process ends, otherwise proceeds to the next step. The power transmission equipment includes at least one of the following: composite insulator core rods, porcelain insulator steel caps, and crimping hardware. For suspected heating defects identified, the relevant responsible persons are notified by platform messages, emails, and text messages for manual review. The relevant responsible persons review the current image on the platform, which can be reviewed through experience or manual temperature measurement. If the manual review determines that it is a defect, a first-line review work order is generated, otherwise the typical equipment defect identification sub-process is entered for defect identification and defect rule library update and optimization. After receiving the review work order, the first-line professionals review the corresponding power transmission equipment in accordance with relevant requirements. If the review confirms that it is a defect, the relevant responsible persons are notified to perform relevant repairs and equipment replacements, otherwise the review image and results are fed back to the platform side, and the platform iteratively updates the defect recognition model and defect rule library based on the feedback image.

[0148] like Fig.13As shown in FIG, the process of infrared image quality evaluation is described in detail. The infrared temperature analysis tool is used to read the temperature information of the infrared image and determine whether the temperature information is valid. If it cannot be read, the image is judged to be unqualified. If it can be read, the next step is performed. The grayscale variance, Sobel gradient, Roberts gradient, Laplace gradient, energy gradient and fast Fourier transform domain values ​​of the infrared image are calculated as the fuzzy evaluation attributes of the infrared image. The support vector machine model is used to perform binary classification on these six attributes. The process of training the support vector machine binary classification model may include: preparing a training set and annotating it, dividing the training set into clear and fuzzy Classification, clarity and blur are provided in a 1:1 ratio; use the six fuzzy evaluation attributes mentioned above as the input of the support vector machine model, and train the binary classification model until the requirements are met, stop training and save the model; use the trained support vector machine model to perform fuzzy evaluation on the infrared image. If it is judged to be fuzzy, the infrared image is unqualified. If it is judged to be clear, proceed to the next step; call the pre-trained device background determination model to determine whether the background of the infrared image is the sky. If it is not the sky, the image is unqualified. If it is the sky, proceed to the next step. The device background determination model training process may include: using the open source labeling tool labelImage to label the infrared image. The image is annotated, and the labels are divided into sky and not sky. The annotated range is the insulator area, crimping hardware and other power transmission equipment areas. If the background of the power transmission equipment area is the sky, it is marked as sky, otherwise it is marked as not sky; the target detection algorithm is used to train the model until the requirements are met, stop training and save the model; the trained target detection model is used to recognize the infrared image. If the device with the background of the sky cannot be recognized in the infrared image, the image is unqualified. If the device with the background of the sky is recognized in the infrared image, the next step is carried out; the image segmentation algorithm is used to respectively identify the full view of the composite insulator, the full view of the porcelain insulator, the unobstructed core rod of the composite insulator and the porcelain The insulator does not block the steel cap. The ratio of the unblocked area of ​​the composite insulator core rod to the entire composite insulator and the ratio of the unblocked steel cap of the porcelain insulator to the entire porcelain insulator are calculated respectively to determine whether the composite insulator core rod area and the porcelain insulator steel cap area are blocked. If the ratio is less than 0.5, it is considered that the composite insulator core rod and the porcelain insulator steel cap are blocked, and the infrared image is determined to be qualified. If the ratio is greater than 0.5, the infrared image is determined to be unqualified. Among them, the crimping hardware does not need to calculate the blocking ratio, that is, there is no need to determine whether the crimping hardware is blocked. If the infrared image is unqualified after the above determination, the process ends, otherwise the heating defect identification of the transmission equipment is carried out.

[0149] like Fig.14As shown in the figure, the recognition process of heating defects in power transmission equipment is described in detail. The trained improved Mask RCNN algorithm model is used to segment the infrared image to obtain the accurate segmentation area of ​​the power transmission equipment. According to the segmentation result, the temperature analysis tool for reading the temperature of the drone infrared image is called to read the temperature information in the segmented area, and the highest temperature, lowest temperature, average temperature and relative temperature difference in the segmented area are extracted; according to the extracted temperature information, it is compared with the defined defect rule library of different equipment. If the highest temperature is higher than the highest temperature in the corresponding defect rule library and the relative temperature difference is greater than the relative temperature difference in the rule library, it is determined that the current equipment has a suspected heating risk, otherwise there is no heating risk. Among them, the accurate segmentation image of the crimping hardware is shown in the figure. Fig.15 shown.

[0150] The embodiment of the present invention uses a drone to automatically inspect power transmission equipment to obtain infrared images, performs image quality assessment on the infrared images obtained by the inspection and generates a corresponding image quality evaluation, and performs subsequent heating defect recognition on the infrared images with qualified image quality evaluation, thereby overcoming the defects of incomplete image quality evaluation and low manual recognition rate in the prior art. The Mask RCNN algorithm based on the added attention mechanism performs pixel-level positioning of the power transmission equipment in the qualified infrared image, realizes accurate segmentation of the infrared image, and solves the problem that the prior art cannot accurately locate the power transmission equipment area in the infrared image. Combined with the image processing algorithm and manual review, the accuracy of defect recognition is improved, unnecessary repairs and equipment replacements are reduced, and the maintenance cost is reduced.

[0151] Fig.16 The present invention provides a schematic diagram of a device for identifying heating defects in power transmission equipment. Fig.16 As shown, the device includes.

[0152] The infrared image acquisition module 710 is used to acquire infrared images obtained by inspecting power transmission equipment using a drone.

[0153] The quality evaluation module 720 is used to generate an image quality evaluation based on the image content and temperature data of the infrared image.

[0154] The defect recognition module 730 is used to generate a heating defect recognition result of the power transmission equipment according to the infrared image whose image quality evaluation meets the preset conditions.

[0155] In the embodiment of the present invention, an infrared image obtained by a drone inspecting a power transmission device is obtained, the image content of the infrared image is detected from the acquired infrared image, and the temperature data of the infrared image is extracted, and the infrared image is analyzed and evaluated based on the temperature data and the image content, respectively, and the result of the analysis and evaluation is used as an image quality evaluation, and the generated quality evaluation is screened according to a preset condition, and the infrared image corresponding to the preset condition is identified for heating defects, and the infrared image is used to determine whether the power transmission device has a heating defect and output the heating defect identification result. In the embodiment of the present invention, the heating defect of the power transmission device is identified based on the image content and temperature data of the infrared image, which overcomes the defects of the prior art in that the image quality evaluation is not comprehensive and the manual recognition rate is low, so that the missed alarm rate and the false alarm rate of the heating defect identification of the power transmission device are reduced, and the infrared image corresponding to the image quality evaluation that meets the preset conditions is used for subsequent heating defect identification of the power transmission device, which solves the problem that the prior art cannot accurately locate the power transmission device area in the infrared image, so that the accuracy rate of the heating defect identification of the power transmission device is improved, and the beneficial effect of improving the efficiency of the heating defect identification of the power transmission device is achieved.

[0156] Furthermore, the quality evaluation module 720 also includes: a temperature data evaluation submodule, which is used to call a preset temperature analysis tool, extract the temperature data of the infrared image, and generate a first image quality evaluation of the infrared image based on the reading result of the temperature data; an image content evaluation submodule, which is used to call a pre-trained image content analysis model to evaluate the image content of the infrared image and obtain a second image quality evaluation of the infrared image.

[0157] Furthermore, the image content evaluation submodule also includes: a fuzzy parameter extraction unit, which is used to extract fuzzy parameters of the infrared image as image content, call the image content analysis model to classify the fuzzy parameters, and obtain the image fuzzy classification result as the second image quality evaluation; an image background detection unit, which is used to call the image content analysis model to detect the specific situation of the image background including the sky in the image content, and obtain the second image quality evaluation of the infrared image; an image content unit, which is used to call the image content analysis model to detect the occlusion of the power transmission equipment in the image content, and obtain the second image quality evaluation of the infrared image.

[0158] Furthermore, the image content unit also includes: an image segmentation subunit, which is used to obtain an image segmentation algorithm of an image content analysis model, and call the image segmentation algorithm to extract a segmented image corresponding to the power transmission equipment in the infrared image; wherein the image segmentation algorithm at least includes a mask area convolutional neural network with an attention mechanism added; an occlusion ratio calculation subunit, which is used to obtain an occlusion ratio calculation algorithm of the image content analysis model, and count the occlusion ratio of each segmented image according to the occlusion ratio calculation algorithm; an image evaluation subunit, which is used to generate a second image with a qualified image quality evaluation if the occlusion ratio is greater than or equal to a threshold ratio, and to generate a second image with an unqualified image quality evaluation if the occlusion ratio is less than the threshold ratio.

[0159] Furthermore, the defect identification module 730 also includes: an image comprehensive evaluation submodule, which is used to determine that the first image quality evaluation corresponding to the temperature data in the image quality evaluation and the second image quality evaluation corresponding to the image content are both qualified images, and then determine that the image quality evaluation meets the preset conditions; a temperature data extraction submodule, which is used to extract key point temperature data corresponding to the image area of ​​the power transmission equipment in the infrared image; a defect matching submodule, which is used to match the defect rules of the corresponding power transmission equipment in the preset defect rule library according to the key point temperature data, wherein the defect rules include the maximum temperature dividing line value and the relative temperature difference dividing line value of the power transmission equipment; a defect result generation submodule, which is used to use the heating defect associated with the defect rule as the heating defect identification result of the power transmission equipment.

[0160] Furthermore, the temperature data extraction submodule also includes: an image segmentation algorithm acquisition unit, used to obtain a pre-trained image segmentation algorithm, wherein the image segmentation algorithm at least includes a mask area convolutional neural network with an attention mechanism added; an image area acquisition unit, used to call the image segmentation algorithm to segment the infrared image to obtain an image area of ​​the power transmission equipment, wherein the image area at least includes a composite insulator core rod area, a porcelain insulator steel cap area, and a crimping hardware area; a temperature data extraction unit, used to extract temperature data of at least one preset key point in each image area as key point temperature data.

[0161] Optionally, the fuzzy parameter extraction unit is specifically used to: obtain a pre-trained support vector machine model as an image content analysis model, wherein the training data set of the support vector machine model includes at least a clear image data set and a blurred image data set; extract fuzzy parameters of the infrared image, wherein the fuzzy parameters include at least one of the following: grayscale variance, Sobel gradient, Roberts gradient, Laplace gradient, energy gradient and FFT transform domain value; input the fuzzy parameters into the support vector machine model to obtain an image fuzzy classification result of the support vector machine model corresponding to the infrared image, if the image fuzzy classification result is that the image is clear, then a second image with a qualified image quality evaluation is generated, and if the image fuzzy classification result is that the image is blurred, then a second image with an unqualified image quality evaluation is generated.

[0162] Optionally, the image background detection unit is specifically used to: obtain a pre-trained target detection model as an image content analysis model, wherein the training data set of the target detection model includes at least labeled images of power transmission equipment, and the labeled information of the training data set includes sky and non-sky; input the image content into the image content analysis model to obtain a sky recognition result of the image background; if the sky recognition result is that the sky is recognized, then a second image with a qualified image quality evaluation is generated; if the sky recognition result is that the sky is not recognized, then a second image with an unqualified image quality evaluation is generated.

[0163] Optionally, the occlusion ratio calculation subunit is specifically used to: count the first minimum circumscribed rectangle of the entire composite insulator in the segmented image and the second minimum circumscribed rectangle of the unobstructed core rod, and use the first ratio of the rectangular long side of the second minimum circumscribed rectangle to the rectangular long side of the first minimum circumscribed rectangle as the occlusion ratio; count the third minimum circumscribed rectangle of the entire porcelain insulator in the segmented image and the unobstructed length of the porcelain insulator steel cap, and use the second ratio of the unobstructed length to the rectangular long side of the third minimum circumscribed rectangle as the occlusion ratio; wherein the unobstructed length can be determined by the segmentation boundary box of the porcelain insulator steel cap and the pixel points of the porcelain insulator steel cap.

[0164] Optionally, the image segmentation algorithm acquisition unit includes: a mask area convolutional neural network with an added attention mechanism includes at least: a backbone network, a region proposal network, an interest region pooling layer, and a Head network, wherein the backbone network, the region proposal network, and the Head network are respectively added with convolutional attention modules.

[0165] Optionally, the temperature data extraction unit is specifically used to: collect normal infrared image samples and heating defect infrared image samples of different types of power transmission equipment; extract temperature data of the image key equipment area corresponding to the power transmission equipment in the normal infrared image samples and the heating defect infrared image samples, wherein the temperature data includes at least one of the following: maximum temperature, minimum temperature, average temperature and relative temperature difference; for different types of power transmission equipment, compare the temperature data of each image key equipment area of ​​the normal infrared image samples and the heating defect infrared image samples to obtain the maximum temperature and relative temperature difference dividing line values ​​of the corresponding image key equipment area, wherein the image key equipment area includes at least one of the following: composite insulator core rod area, porcelain insulator steel cap area and crimping hardware area; store the image key equipment area of ​​each power transmission equipment and the dividing line value of the image key equipment area as defect rules of a preset defect rule library.

[0166] Embodiments of the present invention provide an electronic device, a computer-readable storage medium and a computer program product for a method for identifying heating defects of power transmission equipment.

[0167] Fig.17 A schematic diagram of an electronic device that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown in the embodiments of the present invention, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of the present invention described and / or required herein.

[0168] like Fig.17 As shown, the electronic device includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the ROM 12 or the computer program loaded from the storage unit 18 to the RAM 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0169] A number of components in the electronic device are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0170] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit, a graphics processing unit, various special artificial intelligence computing chips, various processors running machine learning model algorithms, digital signal processors, and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, for example, a method for identifying heating defects of power transmission equipment.

[0171] In some embodiments, the method for identifying a heating defect of a power transmission device may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for identifying a heating defect of a power transmission device may be performed. Alternatively, in other embodiments, the processor 11 may be configured as the method for identifying a heating defect of a power transmission device in any other appropriate manner (for example, by means of firmware).

[0172] Various implementations of the systems and techniques described above in the embodiments of the present invention may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays, application specific integrated circuits, application specific standard products, systems on a chip, load programmable logic devices, computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: being implemented in one or more computer programs, which may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general programmable processor, which may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0173] The computer programs for implementing the methods of the embodiments of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer programs are executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer programs may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0174] In the context of an embodiment of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0175] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a cathode ray tube or a liquid crystal display monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including: acoustic input, voice input, or tactile input).

[0176] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area networks, wide area networks, blockchain networks, and the Internet.

[0177] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private server services.

[0178] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0179] The above specific implementations do not constitute a limitation on the protection scope of the embodiments of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for identifying heating defects in power transmission equipment, characterized in that: The method comprises: Obtain infrared images of power transmission equipment inspected by drones; generating an image quality assessment based on image content and temperature data of the infrared image; The infrared image that satisfies a preset condition according to the image quality evaluation generates a heating defect recognition result of the power transmission equipment.

2. The method according to claim 1, characterized in that: The generating of the image quality evaluation based on the image content and temperature data of the infrared image comprises: Calling a preset temperature analysis tool to extract the temperature data of the infrared image, and generating a first image quality evaluation of the infrared image according to a reading result of the temperature data; A pre-trained image content analysis model is called to evaluate the image content of the infrared image to obtain a second image quality evaluation of the infrared image.

3. The method according to claim 2, characterized in that: The calling of a pre-trained image content analysis model to evaluate the image content of the infrared image to obtain a second image quality evaluation of the infrared image includes: extracting fuzzy parameters of the infrared image as the image content, calling the image content analysis model to classify the fuzzy parameters, and obtaining an image fuzzy classification result as the second image quality evaluation; calling the image content analysis model to detect a specific situation of an image background including a sky in the infrared image as the image content, and obtaining a second image quality evaluation of the infrared image according to the image content; The image content analysis model is called to detect the occlusion of the power transmission equipment in the infrared image as the image content, and a second image quality evaluation of the infrared image is obtained according to the image content.

4. The method according to claim 3, characterized in that: The step of extracting the fuzzy parameters of the infrared image as the image content, calling the image content analysis model to classify the fuzzy parameters, and obtaining the image fuzzy classification result as the second image quality evaluation includes: Acquire a pre-trained support vector machine model as the image content analysis model, wherein the training data set of the support vector machine model includes at least a clear image data set and a blurred image data set; Extracting the fuzzy parameters of the infrared image, wherein the fuzzy parameters include at least one of the following: grayscale variance, Sobel gradient, Roberts gradient, Laplace gradient, energy gradient and fast Fourier transform domain value; Inputting the fuzzy parameter into the support vector machine model to obtain the image fuzzy classification result of the support vector machine model corresponding to the infrared image; If the image blur classification result is that the image is clear, then an image with the second image quality evaluation as qualified is generated; if the image blur classification result is that the image is blurred, then an image with the second image quality evaluation as unqualified is generated.

5. The method according to claim 3, characterized in that: The calling of the image content analysis model to detect the specific situation that the image background in the infrared image includes the sky as the image content, and obtaining the second image quality evaluation of the infrared image according to the image content, comprises: Acquire a pre-trained target detection model as the image content analysis model, wherein a training data set of the target detection model includes at least an annotated image of power transmission equipment, and the annotated information of the training data set includes sky and non-sky; Inputting the infrared image into the image content analysis model to obtain a sky recognition result of the image background in the infrared image; If the sky recognition result is that the sky is recognized, an image with a qualified second image quality evaluation is generated; if the sky recognition result is that the sky is not recognized, an image with an unqualified second image quality evaluation is generated.

6. The method according to claim 3, characterized in that: The calling of the image content analysis model to detect the occlusion of the power transmission equipment in the infrared image as the image content, and obtaining a second image quality evaluation of the infrared image according to the image content, comprises: Obtaining an image segmentation algorithm of the image content analysis model, and calling the image segmentation algorithm to extract a segmented image corresponding to the power transmission device in the infrared image; wherein the image segmentation algorithm at least includes a mask region convolutional neural network with an attention mechanism added; Obtaining an occlusion ratio calculation algorithm of the image content analysis model, and counting the occlusion ratio of each of the segmented images according to the occlusion ratio calculation algorithm; If the occlusion ratio is greater than or equal to the threshold ratio, an image with a qualified second image quality evaluation is generated; if the occlusion ratio is less than the threshold ratio, an image with an unqualified second image quality evaluation is generated.

7. The method according to claim 6, characterized in that: The counting of the occlusion ratios of the segmented images according to the occlusion ratio calculation algorithm includes at least one of the following: Counting the first minimum circumscribed rectangle of the whole composite insulator in the segmented image and the second minimum circumscribed rectangle of the unobstructed core rod, and taking a first ratio of the long side of the second minimum circumscribed rectangle to the long side of the first minimum circumscribed rectangle as the obstruction ratio; Counting the third minimum circumscribed rectangle of the whole image of the porcelain insulator and the unobstructed length of the porcelain insulator steel cap in the segmented image, and taking the second ratio of the unobstructed length to the rectangular long side of the third minimum circumscribed rectangle as the obstruction ratio; The unobstructed length may be determined by a segmentation boundary box of the porcelain insulator steel cap and pixel points of the porcelain insulator steel cap.

8. The method according to claim 1, characterized in that: The step of generating a heating defect recognition result of the power transmission equipment according to the infrared image that satisfies a preset condition according to the image quality evaluation includes: Determining that the first image quality evaluation corresponding to the temperature data in the image quality evaluation and the second image quality evaluation corresponding to the image content are both qualified images, then determining that the image quality evaluation meets the preset condition; Extracting temperature data of key points corresponding to the image area of ​​the power transmission equipment in the infrared image; According to the key point temperature data, a defect rule corresponding to the power transmission equipment is matched in a preset defect rule library; The heating defect associated with the defect rule is used as the heating defect identification result of the power transmission equipment.

9. The method according to claim 8, characterized in that: The step of extracting temperature data of key points corresponding to the image area of ​​the power transmission device in the infrared image includes: Obtaining a pre-trained image segmentation algorithm, wherein the image segmentation algorithm at least includes a mask region convolutional neural network with an attention mechanism added; Calling the image segmentation algorithm to segment the infrared image to obtain the image area of ​​the power transmission equipment, wherein the image area at least includes a composite insulator core rod area, a porcelain insulator steel cap area, and a crimping hardware area; The temperature data of at least one preset key point in each of the image regions is respectively extracted as the key point temperature data.

10. The method according to claim 9, characterized in that: The masked region convolutional neural network with added attention mechanism includes at least: a backbone network, a region proposal network, a region of interest pooling layer, and a Head network, wherein the backbone network, the region proposal network, and the Head network are respectively added with convolutional attention modules.

11. The method according to claim 8, characterized in that: The construction process of the preset defect rule base includes: Collecting normal infrared image samples and heating defect infrared image samples of different types of the power transmission equipment; Extracting temperature data of a key device area of ​​the image corresponding to the power transmission device in the normal infrared image sample and the heating defect infrared image sample; For different types of power transmission equipment, compare the temperature data of each of the key image equipment areas of the normal infrared image samples and the heating defect infrared image samples to obtain the highest temperature and the boundary value of the relative temperature difference corresponding to the key image equipment area; The image key device area of ​​each of the power transmission devices and the boundary value of the image key device area are stored as defect rules of the preset defect rule library.

12. The method according to claim 11, characterized in that: The temperature data includes at least one of the following: maximum temperature, minimum temperature, average temperature and relative temperature difference.

13. The method according to claim 11, characterized in that: The image key equipment area includes at least one of the following: a composite insulator core rod area, a porcelain insulator steel cap area, and a crimping hardware area.

14. The method according to claim 8, characterized in that: The defect rule includes a maximum temperature boundary value and a relative temperature difference boundary value of the power transmission equipment.

15. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for identifying heating defects of power transmission equipment according to any one of claims 1-14.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for identifying heating defects of power transmission equipment according to any one of claims 1 to 14 when executed.

Images