A composite insulator rod core heating infrared image defect training sample generation method

Abstract

The present application relates to overhead power line operation maintenance and detection repair technical field, disclose a kind of composite insulator rod core heating infrared image defect training sample generation method, comprising: from the original defect-free image in defect-free inspection infrared sample library is taken out and is converted into normalized gray image by image preprocessing operation;The heating point of composite insulator rod core in normalized gray image is positioned, and heating point pixel information is obtained;Generate heating area mask on original defect-free image based on heating point pixel information using region growing algorithm;Original defect-free image is synthesized using heating area mask for heating effect, and defect simulation image is generated;Calculate the circumscribed rectangle frame of heating area mask, generate the defect label of defect simulation image, according to defect simulation image and its defect label, generate defect training sample. Expand existing unmanned aerial vehicle power inspection infrared image data sample set.

Description

Technical Field

[0001] This invention relates to the field of overhead power line operation, maintenance, inspection and repair technology, and more specifically, to a method for generating training samples of defects in infrared images of composite insulator core heating. Background Technology

[0002] With the booming development of the national economy and the continuous rise in electricity demand, my country's power transmission lines have undergone unprecedented expansion. To ensure the safety and stability of the entire power system, drones are used for routine inspections of key components of power equipment on overhead transmission lines and towers. During the inspection missions, the image data captured by the drones is transmitted back to the workstation in real time, where staff then use intelligent algorithms to detect defects in the basic components of the transmission lines. Although these intelligent detection algorithms have made significant progress in improving inspection efficiency and accuracy, in actual operation and maintenance, some types of defects occur infrequently, resulting in a severe shortage of samples available for algorithm training. This directly restricts further improvement in the detection accuracy of related algorithms. For example, the core heating defect of composite insulators is one such defect with few samples. As a key component in transmission lines, the reliability of composite insulators is directly related to the safe operation of the power grid. Common composite insulator fault modes include insulation sheath damage, core breakage, and interface breakdown. Core heating is an important type of fault, usually caused by internal defects, moisture intrusion, or partial discharge at the core-sheath interface. Core heating accelerates the aging of insulation materials, reduces the mechanical strength of insulators, and in severe cases, may lead to insulator breakage and power outages. Infrared thermal imaging technology is an effective means of detecting heating defects in composite insulators. However, because core heating defects occur relatively infrequently in actual operation, it is difficult to obtain sufficient real-world defect samples for training AI models. This results in a limited number of core heating defect samples during routine inspections, hindering the improvement of detection accuracy for this type of defect in current defect detection and identification algorithms. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a method for generating training samples of defects in infrared images of composite insulator rod core heating.

[0004] According to one aspect of the present invention, a method for generating training samples of defects in infrared images of composite insulator core heating is provided, comprising:

[0005] Take an original, defect-free image from the defect-free inspection infrared sample library, and perform image preprocessing on the original, defect-free image to convert it into a normalized grayscale image.

[0006] The heating points of the composite insulator rod core in the normalized grayscale image are located and the pixel information of the heating points is obtained.

[0007] A region growing algorithm is used to generate a heat-generating region mask on the original defect-free image based on the pixel information of heat-generating points.

[0008] By using a mask of the heating area, the original defect-free image is synthesized to create a heating effect, thus generating a defect-simulated image.

[0009] Calculate the bounding rectangle of the heat-generating region mask, generate defect annotations for the defect simulation image, and generate defect training samples based on the defect simulation image and its defect annotations.

[0010] Optionally, the heating points of the composite insulator core in the normalized grayscale image are located to obtain the location information of the heating points, including:

[0011] Instance segmentation is performed on the normalized grayscale image, and the mask regions of large hardware fittings and composite insulators are selected from the segmented instances.

[0012] Calculate the adjacent pixels of the large fitting mask area and the composite insulator mask area to determine the pixel information of the heat source.

[0013] Optionally, the normalized grayscale image is segmented into instances, and the large metal fitting mask region and the composite insulator mask region are selected from the segmented instances, including:

[0014] The backbone network is used to extract image features from the normalized grayscale image to obtain the feature image;

[0015] A region proposal network is used to generate multiple candidate RoIs on the feature image;

[0016] Candidate RoIs of different sizes are converted into RoI feature maps of a fixed size;

[0017] Classify RoI feature maps and precisely adjust bounding boxes to obtain RoI images of multi-class instances;

[0018] The FCN network is used to segment the pixels in the RoI image of multi-class instances to generate a binary mask for multi-class instances;

[0019] Screening large fitting mask regions and composite insulator mask regions from binary masks of multiple categories of instances.

[0020] Optionally, the adjacent pixels of the large fitting mask area and the composite insulator mask area are calculated to determine the hot spot pixel information, including:

[0021] Traverse all pixels in the composite insulator mask region and determine whether the 8-neighborhood of each pixel contains pixels in the large fitting mask region. If so, determine it as an adjacent pixel.

[0022] Determine the pixel information of the hot spot based on all adjacent pixels traversed.

[0023] Optionally, a heat-generating region mask is generated on the original defect-free image based on the pixel information of the heat-generating points using a region growing algorithm, including:

[0024] Calculate the center point coordinates of the geometric center of all hot spot pixels based on the hot spot pixel information;

[0025] The center point coordinates are used as a seed input to the region growing algorithm to generate a mask of the heated region on the original defect-free image.

[0026] Optionally, the center point coordinates are used as a seed input to a region growing algorithm to generate a heat-generating region mask on the original defect-free image, including:

[0027] Using the input seed as a reference point, the similarity between the candidate pixels in the 8-neighborhood of the reference point and the features of the current region is calculated. Candidate pixels that meet the preset similarity threshold are included in the current region to generate a heat-generating region mask.

[0028] Optionally, a heating effect is synthesized from the original defect-free image using a heating area mask to generate a defect simulation image, including:

[0029] Iterate through each pixel in the heat-generating area mask and find the distance C from the center point in the heat-generating area mask. x C , y C The farthest point D x D , y D );

[0030] Based on the center point C ( x C , y C ) and the point D that is farthest from it. x D , y D ), calculate M( ) for each pixel in the heat-generating area mask. x M , y M Weighting factors w ( x M , y M ):

[0031]

[0032] In the formula, the parameter It is a random number that follows a uniform distribution;

[0033] Based on the heat-generating region mask, each pixel M( x M , y M Weighting factors w ( x M , y M The original defect-free image and the mask of the heated area are used to synthesize the heating effect to generate a defect simulation image I. S Among them, the defect simulation image I S The expression is:

[0034]

[0035] In the formula, I represents the original defect-free image; M Masking for the heat-generating area; w , where is the weight factor for each pixel in the mask of the heat-generating region.

[0036] According to another aspect of the present invention, a device for generating training samples of infrared images of defects in the heating element of a composite insulator rod is provided, comprising:

[0037] The conversion module is used to retrieve an original defect-free image from the defect-free inspection infrared sample library and perform image preprocessing operations on the original defect-free image to convert it into a normalized grayscale image.

[0038] The positioning module is used to locate the heating points of the composite insulator rod core in the normalized grayscale image and obtain the pixel information of the heating points.

[0039] The first generation module is used to generate a heat-generating region mask on the original defect-free image based on the heat-generating point pixel information using a region growing algorithm.

[0040] The second generation module is used to synthesize the heating effect of the original defect-free image using a heating area mask to generate a defect simulation image.

[0041] The third generation module is used to calculate the bounding rectangle of the heat-generating area mask, generate defect annotations for the defect simulation image, and generate defect training samples based on the defect simulation image and its defect annotations.

[0042] According to another aspect of the present invention, a computer-readable storage medium is provided, the storage medium storing a computer program for performing the methods described in any of the above aspects of the present invention.

[0043] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the method described in any of the preceding aspects of the present invention.

[0044] Therefore, this invention can utilize existing defect-free UAV power line inspection infrared image data to generate composite insulator core heating defect samples, providing defect simulation training samples for the development of defect detection algorithms, expanding the existing UAV power line inspection infrared image data sample set, thereby improving the detection accuracy of defect detection algorithms for composite insulator core heating defects, and enhancing the generalization performance of the algorithm in different scenarios. Attached Figure Description

[0045] Exemplary embodiments of the present invention can be more fully understood by referring to the following figures:

[0046] Figure 1 This is a flowchart illustrating a method for generating training samples of infrared images showing defects in the heating of composite insulator rod cores, provided in an exemplary embodiment of the present invention.

[0047] Figure 2 This is a flowchart illustrating the overall process of generating infrared image training samples of heating defects in composite insulator rod cores, provided by an exemplary embodiment of the present invention.

[0048] Figure 3 This is an example diagram of an infrared camera sample provided in an exemplary embodiment of the present invention;

[0049] Figure 4 Yes Figure 3 The result of instance segmentation is that the instance segmentation mask is superimposed on the... Figure 3 Above; the red area in the diagram is the composite insulator, the yellow area is the large fittings, and the green, blue, and light blue areas are the conductors and ground wires;

[0050] Figure 5 This is a schematic diagram of the mask image of the heating area after processing by the mask processing algorithm provided in an exemplary embodiment of the present invention;

[0051] Figure 6 It is to utilize Figure 5 Mask pair Figure 3 The final composite insulator core heating defect simulation sample is obtained by synthesizing the heating effect of the original defect-free inspection sample.

[0052] Figure 7 This is a schematic diagram of the structure of a composite insulator core heating infrared image defect training sample generation device provided in an exemplary embodiment of the present invention;

[0053] Figure 8 This is the structure of an electronic device provided in an exemplary embodiment of the present invention. Detailed Implementation

[0054] Hereinafter, exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein.

[0055] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention.

[0056] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of the present invention are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0057] It should also be understood that in the embodiments of the present invention, "multiple" can refer to two or more, and "at least one" can refer to one, two or more.

[0058] It should also be understood that any component, data or structure mentioned in the embodiments of the present invention can generally be understood as one or more unless explicitly defined or given contrary instructions in the context.

[0059] Furthermore, the term "and / or" in this invention is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this invention generally indicates that the preceding and following related objects have an "or" relationship.

[0060] It should also be understood that the description of the various embodiments in this invention emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0061] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0062] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0063] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, they should be considered part of the specification.

[0064] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0065] The embodiments of this invention can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate together with a wide range of other general-purpose or special-purpose computing system environments or configurations. Well-known examples of terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0066] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0067] Exemplary method:

[0068] Figure 1 This is a schematic flowchart illustrating a method for generating training samples of infrared images showing defects in the heating element of a composite insulator rod, provided in an exemplary embodiment of the present invention. This embodiment can be applied to electronic devices, such as… Figure 1 As shown, the method 100 for generating training samples of infrared images showing defects in the heating element of a composite insulator rod includes the following steps:

[0069] Step 101: Take an original defect-free image from the defect-free inspection infrared sample library, and perform image preprocessing on the original defect-free image to convert it into a normalized grayscale image.

[0070] Step 102: Locate the heating points of the composite insulator rod core in the normalized grayscale image and obtain the pixel information of the heating points;

[0071] Step 103: Use the region growing algorithm to generate a heat-generating region mask on the original defect-free image based on the heat-generating point pixel information;

[0072] Step 104: Use a heat-generating area mask to synthesize the heat-generating effect of the original defect-free image to generate a defect simulation image;

[0073] Step 105: Calculate the bounding rectangle of the heat-generating area mask, generate defect annotations for the defect simulation image, and generate defect training samples based on the defect simulation image and its defect annotations.

[0074] Specifically, this invention addresses the problem of limited sample numbers of composite insulator core heating defects during routine inspections, which hinders the improvement of detection accuracy for this type of defect in current defect detection and identification algorithms. It proposes a method to synthesize high-quality composite insulator core heating defect samples using existing defect-free inspection infrared samples through a series of image processing steps. Furthermore, it provides a tool for generating defect simulation and synthesis data for training UAV power line inspection defect detection algorithms. This allows for the generation of a large number of composite insulator core heating defect samples using defect-free inspection image samples, thereby expanding the number of such defect samples in the existing training set and improving the detection accuracy of defect detection and identification algorithms for composite insulator core heating defects.

[0075] Furthermore, the overall technical approach adopted in this invention is as follows: Figure 2 As shown: First, a defect-free real inspection image is retrieved from the defect-free inspection infrared sample library and converted into a normalized grayscale image through image preprocessing. Since the heating area of ​​the composite insulator core is often located near the contact point with the large fittings, this invention uses this rule to locate the heating point of the composite insulator core. Then, a heating area mask is generated based on the heating point using a region growing algorithm, and finally, the heating effect is generated within this area. This completes the generation of the heating defect sample, and simultaneously calculates the bounding rectangle of the heating area mask to generate defect annotations. The specific implementation process is as follows:

[0076] 1. Preprocess the defect-free inspection images by converting them to grayscale and normalizing them. Then, perform instance segmentation on the images. Filter the segmented mask results into two categories: large fittings and composite insulators. Calculate the adjacent pixels of these two types of mask regions. The calculation method is: traverse all pixels in the composite insulator mask region and determine whether the 8-neighborhood of a pixel contains pixels in the large fitting mask region. If so, it is determined as a contact point.

[0077] Furthermore, instance segmentation aims to identify each instance of interest in an image and generate a pixel-level segmentation mask for each instance. In this invention, the instance segmentation network can be, but is not limited to, Mask R-CNN. The algorithm's principle is as follows: First, a backbone network (such as ResNet) is used to extract image features. A Region Proposal Network (RPN) generates candidate RoIs on the feature map, and then RoIs of different sizes are converted into feature maps of a fixed size while maintaining spatial alignment accuracy. The RoIs are classified, and their bounding boxes are precisely adjusted. Finally, a small FCN network segments the pixels within the RoI, outputting a binary mask for the instance. This neural network uses a loss function that jointly optimizes the classification loss, bounding box regression loss, and binary mask loss. In this invention, instance segmentation uses existing UAV power line inspection infrared samples to construct a training set, annotates the masks of each component, and trains the Mask R-CNN network to identify common power components (including glass insulators, porcelain insulators, composite insulators, hardware, conductors, towers, etc.) in infrared images and generate instance masks for each component.

[0078] 2. Calculate the coordinates C of the geometric center of all contact points. x C , y C The coordinates of the center point C are used as the seed input to the region growing algorithm to generate a new mask on the original inspection image, namely the heat-generating region mask.

[0079] Furthermore, the basic principle of the region growing algorithm is a region aggregation method based on pixel similarity. This algorithm extracts similar regions near the input seed point through the following process: First, using the input seed point as a reference point, the algorithm systematically examines pixels within the 8-neighborhood of the reference point. By calculating the similarity between candidate pixels and the features of the current region, pixels that meet a preset similarity threshold are included in the current region. In this invention, a grayscale threshold with respect to the reference point is used as the criterion for determining whether a pixel is included in the current region. Newly added pixels become reference points for subsequent growth, and the algorithm continuously expands outward using a breadth-first or depth-first strategy. This process forms an iterative loop, continuously absorbing neighboring pixels that meet the conditions, achieving gradual growth of the region. When all neighboring pixels no longer meet the similarity conditions, or reach the preset region boundary (such as encountering a significant edge or exceeding the maximum region size), the growth process of the region is complete. Finally, all connected similar pixels will constitute a complete uniform region. This seed-diffusion-based growth mechanism can effectively divide an image into multiple connected regions with uniform characteristics.

[0080] Furthermore, the specific algorithm for synthesizing the heating effect from the original defect-free image using a heating region mask is as follows:

[0081] Iterate through each pixel in the mask and find the coordinates C( distance from the center point) in the mask. x C , y C The farthest point D x D , y D );

[0082] 1) Calculate M(m, m) for each pixel in the mask. x M , y M Weighting factors:

[0083]

[0084] Among them, parameters It is a random number that follows a uniform distribution.

[0085] 2) Assume the original defect-free image is I, and the mask for the heated area is I. M The defect simulation image is I S The pixel values ​​of the above images have all been normalized to (0, 1). The algorithm for synthesizing the heating effect is as follows:

[0086]

[0087] Finally, based on this mask, a bounding rectangle is generated and saved as a labeling file for the defect location.

[0088] In an exemplary embodiment of the present invention, Figures 3-6 The process of generating a simulated sample of heating defects in a composite insulator core from defect-free UAV inspection infrared images is explained in detail below:

[0089] (1) Using infrared image power component instance segmentation network to Figure 3 Perform instance segmentation to obtain the segmentation results. Figure 4 ;

[0090] (2) To Figure 4 The mask regions in the process are screened to identify two categories: large fittings and composite insulators. The contact points of these two regions are calculated, and then the geometric center C of the contact points is calculated.

[0091] (3) Using point C as the seed, the region growing algorithm is used in... Figure 3 Regenerate a new mask Figure 4 ;

[0092] (4) Traverse the heat-generating area mask Figure 5Find the point D that is farthest from point C, and apply the above-mentioned heating effect synthesis algorithm to obtain the final defect simulation image as shown. Figure 6 ;

[0093] (5) Masking the heating area Figure 5 Calculate the bounding rectangle of the annotations circumscribed in the mask and save it as an annotation file.

[0094] Therefore, this invention proposes a method to generate thermal defects in composite insulator cores from existing defect-free UAV power line inspection infrared image data. This method involves a series of image processing steps, including image preprocessing, hot spot localization, hot area generation, and thermal effect generation, which addresses the problem of the limited number of such defects in current inspection samples. Secondly, based on the locations where thermal defects in composite insulator cores are prone to occur, this invention proposes a method to screen for hot areas from defect-free images and designs a thermal effect synthesis algorithm with random control parameters to simulate and generate the thermal effect. Furthermore, it should be noted that the Mask R-CNN algorithm used in this invention can also be replaced by other instance segmentation algorithms.

[0095] In summary, this invention can utilize existing defect-free UAV power line inspection infrared image data to generate composite insulator core heating defect samples, providing defect simulation training samples for the development of defect detection algorithms, expanding the existing UAV power line inspection infrared image data sample set, thereby improving the detection accuracy of defect detection algorithms for composite insulator core heating defects, and enhancing the generalization performance of the algorithm in different scenarios.

[0096] Exemplary device:

[0097] Figure 7 This is a schematic diagram of the structure of a composite insulator core heating infrared image defect training sample generation device provided in an exemplary embodiment of the present invention. Figure 7 As shown, the device 700 includes:

[0098] The conversion module 710 is used to retrieve an original defect-free image from the defect-free inspection infrared sample library and perform image preprocessing operations on the original defect-free image to convert it into a normalized grayscale image.

[0099] The positioning module 720 is used to locate the heating point of the composite insulator rod core in the normalized grayscale image and obtain the pixel information of the heating point.

[0100] The first generation module 730 is used to generate a heat-generating region mask on the original defect-free image based on the heat-generating point pixel information using a region growing algorithm.

[0101] The second generation module 740 is used to synthesize the heating effect of the original defect-free image using a heating area mask to generate a defect simulation image.

[0102] The third generation module 750 is used to calculate the bounding rectangle of the heat-generating area mask, generate defect annotations for the defect simulation image, and generate defect training samples based on the defect simulation image and its defect annotations.

[0103] Optionally, the positioning module 720 includes:

[0104] The segmentation submodule is used to perform instance segmentation on the normalized grayscale image and filter the large hardware mask region and composite insulator mask region from the segmented instances.

[0105] The first calculation submodule is used to calculate the adjacent pixels of the large fitting mask area and the composite insulator mask area to determine the pixel information of the heat point.

[0106] Optionally, the sub-modules are divided, including:

[0107] The extraction unit is used to extract image features from the normalized grayscale image using the backbone network to obtain a feature image.

[0108] The first generation unit is used to generate multiple candidate RoIs on the feature image using a region proposal network;

[0109] The transformation unit is used to convert candidate RoIs of different sizes into RoI feature maps of a fixed size;

[0110] The acquisition unit is used to classify RoI feature maps and precisely adjust bounding boxes to acquire RoI images of multi-class instances;

[0111] The second generation unit is used to segment the pixels in the RoI image of multi-class instances using an FCN network to generate a binary mask for multi-class instances.

[0112] The filtering unit is used to filter the large fitting mask region and the composite insulator mask region from the binary masks of multiple categories of instances.

[0113] Optionally, the computation submodule includes:

[0114] The judgment unit is used to traverse all pixels in the composite insulator mask area and determine whether the 8-neighborhood of each pixel contains pixels in the large fitting mask area. If so, it is determined to be an adjacent pixel.

[0115] The determination unit is used to determine the pixel information of the hot spot based on all traversed adjacent pixels.

[0116] Optionally, the first generation module 730 includes:

[0117] The second calculation submodule is used to calculate the center point coordinates of the geometric center of all hot point pixels based on the hot point pixel information;

[0118] The first generation submodule is used to input the center point coordinates as a seed into the region growing algorithm to generate a heat-generating region mask on the original defect-free image.

[0119] Optionally, the first generation submodule includes:

[0120] The third generation unit is used to calculate the similarity between candidate pixels in the 8-neighborhood of the input seed and the features of the current region, and to include candidate pixels that meet the preset similarity threshold into the current region to generate a heat-generating region mask.

[0121] Optionally, the second generation module 740 includes:

[0122] The traversal submodule is used to traverse each pixel in the heat-generating area mask and find the distance C from the center point in the heat-generating area mask. x C , y C The farthest point D x D , y D );

[0123] The third calculation submodule is used to calculate based on the center point C( x C , y C ) and the point D that is farthest from it. x D , y D ), calculate M( ) for each pixel in the heat-generating area mask. x M , y M Weighting factors w ( x M , y M ):

[0124]

[0125] In the formula, the parameter It is a random number that follows a uniform distribution;

[0126] The second generation submodule is used to generate each pixel M( in the heat-generating region mask). x M , y M Weighting factors w ( x M ,y M The original defect-free image and the mask of the heated area are used to synthesize the heating effect to generate a defect simulation image I. S Among them, the defect simulation image I S The expression is:

[0127]

[0128] In the formula, I represents the original defect-free image; M Masking for the heat-generating area; w , where is the weight factor for each pixel in the mask of the heat-generating region.

[0129] Exemplary electronic device:

[0130] Figure 8 This is the structure of an electronic device provided in an exemplary embodiment of the present invention. For example... Figure 8 As shown, the electronic device 80 includes one or more processors 81 and memory 82.

[0131] The processor 81 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0132] The memory 82 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 81 may execute the program instructions to implement the methods of the software programs of the various embodiments of the present invention described above, and / or other desired functions. In one example, the electronic device may also include an input device 83 and an output device 84, these components being interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0133] In addition, the input device 83 may also include, for example, a keyboard, a mouse, etc.

[0134] The output device 84 can output various information to the outside. The output device 84 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0135] Of course, for the sake of simplicity, Figure 8Only some of the components of this electronic device relevant to the present invention are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device may include any other suitable components depending on the specific application.

[0136] Exemplary computer program products and computer-readable storage media:

[0137] In addition to the methods and apparatus described above, embodiments of the present invention may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of the present invention described in the "Exemplary Methods" section above.

[0138] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of the present invention. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0139] Furthermore, embodiments of the present invention may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps of the methods according to various embodiments of the present invention described in the "Exemplary Methods" section above.

[0140] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0141] The basic principles of the present invention have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in the present invention are merely examples and not limitations, and should not be considered as essential features of each embodiment of the present invention. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the present invention to the necessity of employing the aforementioned specific details.

[0142] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0143] The block diagrams of devices, systems, devices, and systems involved in this invention are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, systems, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0144] The methods and systems of the present invention may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of the present invention are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, the present invention may also be implemented as a program recorded on a recording medium, the program comprising machine-readable instructions for implementing the methods according to the present invention. Thus, the present invention also covers recording media storing programs for performing the methods according to the present invention.

[0145] It should also be noted that in the systems, apparatus, and methods of the present invention, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered equivalents of the present invention. The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the invention. Therefore, the invention is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0146] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the invention to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for generating training samples of defects in infrared images of composite insulator rod core heating, characterized in that, include: Take an original, defect-free image from the defect-free inspection infrared sample library, and perform image preprocessing on the original, defect-free image to convert it into a normalized grayscale image. The heating points of the composite insulator rod core in the normalized grayscale image are located, and the pixel information of the heating points is obtained; A heat-generating region mask is generated on the original defect-free image based on the heat-generating point pixel information using a region growing algorithm. The original defect-free image is synthesized using the heat-generating area mask to create a simulated defect image; Calculate the bounding rectangle of the heat-generating region mask, generate the defect annotation of the defect simulation image, and generate defect training samples based on the defect simulation image and its defect annotation; The original defect-free image is synthesized using the heat-generating region mask to create a simulated defect image, including: Traverse each pixel in the heat-generating area mask and find the distance from the center point C in the heat-generating area mask. x C , y C The farthest point D x D , y D ); Based on the center point C ( x C , y C ) and the point D that is farthest from it. x D , y D ), calculate M( ) for each pixel in the heat-generating area mask. x M , y M Weighting factors w ( x M , y M ): In the formula, the parameter It is a random number that follows a uniform distribution; Based on each pixel M( in the heat-generating area mask) x M , y M Weighting factors w ( x M , y M The original defect-free image and the heat-generating area mask are combined to create a heat-generating effect, thus generating the defect simulation image I. S The defect simulation image I S The expression is: In the formula, I represents the original defect-free image; M Masking for the heat-generating area; w This represents the weight factor for each pixel in the heat generation area mask.

2. The method according to claim 1, characterized in that, The location of the heating points in the composite insulator core of the normalized grayscale image is determined, and the location information of the heating points is obtained, including: The normalized grayscale image is segmented into instances, and the large hardware mask region and the composite insulator mask region are selected from the segmented instances. Calculate the adjacent pixels of the large hardware mask area and the composite insulator mask area to determine the pixel information of the heat source.

3. The method according to claim 2, characterized in that, The normalized grayscale image is segmented into instances, and the large fitting mask region and composite insulator mask region are selected from the segmented instances, including: The backbone network is used to extract the image features of the normalized grayscale image to obtain the feature image; A region proposal network is used to generate multiple candidate RoIs on the feature image; Candidate RoIs of different sizes are converted into RoI feature maps of a fixed size; The RoI feature maps are classified and the bounding boxes are precisely adjusted to obtain RoI images of multi-class instances; The FCN network is used to segment the pixels in the RoI image of multi-class instances to generate a binary mask for multi-class instances; The large fitting mask region and the composite insulator mask region are selected from the binary masks of multiple categories of instances.

4. The method according to claim 2, characterized in that, Calculate the adjacent pixels of the large hardware mask area and the composite insulator mask area to determine the heat point pixel information, including: Traverse all pixels in the composite insulator mask region and determine whether the 8-neighborhood of each pixel contains pixels in the large fitting mask region. If so, determine it as an adjacent pixel. The heat point pixel information is determined based on all adjacent pixels traversed.

5. The method according to claim 1, characterized in that, Generating a heat-generating region mask on the original defect-free image using a region growing algorithm based on the heat-generating point pixel information includes: Calculate the center point coordinates of the geometric center of all the heated pixel based on the heated pixel information; The coordinates of the center point are used as a seed and input into the region growing algorithm to generate the heat-generating region mask on the original defect-free image.

6. The method according to claim 5, characterized in that, The center point coordinates are used as a seed input into a region growing algorithm to generate the heat-generating region mask on the original defect-free image, including: Using the input seed as a reference point, the similarity between the candidate pixels in the 8-neighborhood of the reference point and the features of the current region is calculated. Candidate pixels that meet the preset similarity threshold are included in the current region to generate the heat generation region mask.

7. A device for generating training samples of defects in infrared images of composite insulator rod core heating, characterized in that, include: The conversion module is used to retrieve an original defect-free image from the defect-free inspection infrared sample library and perform image preprocessing operations on the original defect-free image to convert it into a normalized grayscale image. The positioning module is used to locate the heating points of the composite insulator rod core in the normalized grayscale image and obtain the pixel information of the heating points. The first generation module is used to generate a heat-generating region mask on the original defect-free image based on the heat-generating point pixel information using a region growing algorithm. The second generation module is used to synthesize the heating effect of the original defect-free image using the heating area mask to generate a defect simulation image. The third generation module is used to calculate the bounding rectangle of the heat-generating area mask, generate the defect annotation of the defect simulation image, and generate defect training samples based on the defect simulation image and its defect annotation. The original defect-free image is synthesized using the heat-generating region mask to create a simulated defect image, including: Traverse each pixel in the heat-generating area mask and find the distance from the center point C in the heat-generating area mask. x C , y C The farthest point D x D , y D ); Based on the center point C ( x C , y C ) and the point D that is farthest from it. x D , y D ), calculate M( ) for each pixel in the heat-generating area mask. x M , y M Weighting factors w ( x M , y M ): In the formula, the parameter It is a random number that follows a uniform distribution; Based on each pixel M( in the heat-generating area mask) x M , y M Weighting factors w ( x M , y M The original defect-free image and the heat-generating area mask are combined to create a heat-generating effect, thus generating the defect simulation image I. S The defect simulation image I S The expression is: In the formula, I represents the original defect-free image; M Masking for the heat-generating area; w This represents the weight factor for each pixel in the heat generation area mask.

8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for performing the method described in any one of claims 1-6.

9. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method described in any one of claims 1-6.

Images