A Dual-Mode Defect Identification Method and System for Substation Equipment

Abstract

The present invention discloses a dual-modal defect recognition method and system for substation equipment. The method includes: collecting visible light image samples of transformer oil leakage defects and visible light and infrared dual-modal image samples of substation equipment heating defects; building an improved YOLOv8 instance segmentation model, and using the method of image segmentation and background fusion to expand the defect samples; constructing a fused image dataset through image registration and fusion, building an improved YOLOv8 object detection model, using the transformer oil leakage defect samples and the fused image samples for the training of the improved YOLOv8 object detection model, and then identifying the transformer oil leakage defects and substation equipment heating defects through the trained improved YOLOv8 object detection model; using the mapping method to present the detection results of the heating defects of the fused images in the visible light images. By introducing the method of deep learning into the field of substation equipment operation and maintenance technology, the efficient, accurate and intelligent recognition of the structural defects of substation equipment is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of operation and maintenance of power transmission and transformation equipment, and particularly relates to a bimodal defect recognition method and system for substation equipment. Background Art

[0002] Power equipment in a substation is a core component of the substation, playing the roles of voltage transformation, power distribution, and power protection. However, over time, structural defects and heating defects are likely to occur in substation equipment. Common typical defect types include oil leakage defects in transformers, heating defects in transformer bushings, and heating defects in arresters, etc. Currently, the operation and maintenance work of substation equipment mainly relies on manual inspections and regular maintenance, which not only involves a large workload and low efficiency, but also makes it difficult to achieve real-time monitoring. Summary of the Invention

[0003] The present invention provides a bimodal defect recognition method and system for substation equipment to solve the technical problem that the operation and maintenance work of current substation equipment mainly relies on manual inspections and regular maintenance.

[0004] In a first aspect, the present invention provides a bimodal defect recognition method for substation equipment, including:

[0005] Obtain a first visible light image of an oil leakage defect in a transformer, and perform background fusion and expansion on the first visible light image based on a preset improved YOLOv8 instance segmentation model to obtain a structural defect image dataset of substation equipment;

[0006] Obtain a second visible light image and an infrared image of a heating defect in substation equipment, register and fuse the second visible light image and the infrared image to obtain a fused image dataset;

[0007] Iteratively train a preset improved YOLOv8 object detection model according to the structural defect image dataset of substation equipment and the fused image dataset to obtain a bimodal defect recognition model;

[0008] Obtain a first real-time visible light image of the transformer and a real-time fused image of substation equipment, input the first real-time visible light image and the real-time fused image into the bimodal defect recognition model, and the bimodal defect recognition model outputs an oil leakage defect result corresponding to the first real-time visible light image and a heating defect result corresponding to the real-time fused image, where the real-time fused image is obtained by registering and fusing a second real-time visible light image of a heating defect in substation equipment and a real-time infrared image;

[0009] Map the heating defect result to the second real-time visible light image according to the mapping method to obtain the position of the infrared heating defect of the substation equipment.

[0010] In a second aspect, the present invention provides a dual-modal defect recognition system for substation equipment, including:

[0011] An expansion module configured to obtain a first visible light image of the oil leakage defect of the transformer, and perform background fusion expansion on the first visible light image based on a preset improved YOLOv8 instance segmentation model to obtain a structural defect image dataset of the substation equipment;

[0012] A fusion module configured to obtain a second visible light image and an infrared image of the heating defect of the substation equipment, register and fuse the second visible light image and the infrared image to obtain a fused image dataset;

[0013] A training module configured to perform iterative training on a preset improved YOLOv8 object detection model according to the structural defect image dataset of the substation equipment and the fused image dataset to obtain a dual-modal defect recognition model;

[0014] An output module configured to obtain a first real-time visible light image of the transformer and a real-time fused image of the substation equipment, input the first real-time visible light image and the real-time fused image into the dual-modal defect recognition model, and the dual-modal defect recognition model outputs an oil leakage defect result corresponding to the first real-time visible light image and a heating defect result of the real-time fused image, wherein the real-time fused image is obtained by registering and fusing a second real-time visible light image of the heating defect of the substation equipment and a real-time infrared image;

[0015] A mapping module configured to map the heating defect result to the second real-time visible light image according to the mapping method to obtain the position of the infrared heating defect of the substation equipment.

[0016] In a third aspect, an electronic device is provided, which includes: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the steps of the method for recognizing dual-modal defects of substation equipment according to any embodiment of the present invention.

[0017] In a fourth aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program instructions are executed by a processor, the processor is enabled to execute the steps of the method for recognizing dual-modal defects of substation equipment according to any embodiment of the present invention.

[0018] The dual-modal defect recognition method and system for power transformation equipment of the present application collect visible light image samples of transformer oil leakage defects and visible light and infrared dual-modal image samples of power transformation equipment heating defects; in view of the insufficient number of transformer oil leakage defect samples, an improved YOLOv8 instance segmentation model is built, and an image segmentation and background fusion method is used to expand the defect samples; a fused image dataset is constructed through image registration and fusion, and an improved YOLOv8 object detection model is built. The transformer oil leakage defect samples and the fused image samples are used for the training of the improved YOLOv8 object detection model, and then the trained improved YOLOv8 object detection model is used to identify the transformer oil leakage defects and the power transformation equipment heating defects; in view of the problem that the detection results of the power transformation equipment heating defects are difficult to visually present in the operation and maintenance work, a mapping method is used to present the detection results of the heating defects of the fused image in the visible light image. By introducing the deep learning method into the field of power transformation equipment operation and maintenance technology, the efficient, accurate and intelligent identification of the structural defects of power transformation equipment is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0020] Figure 1 It is a flowchart of a method for identifying dual-modal defects of power transformation equipment provided by an embodiment of the present invention;

[0021] Figure 2 It is a flowchart of a method for using multi-intention recognition provided by an embodiment of the present invention;

[0022] Figure 3 It is a flowchart of the training stage of a specific embodiment provided by an embodiment of the present invention;

[0023] Figure 4 It is a structural block diagram of a system for identifying dual-modal defects of power transformation equipment provided by an embodiment of the present invention;

[0024] Figure 5 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] See also Figure 1 , which shows a flow chart of a dual-modal defect identification method for substation equipment of the present application.

[0027] like Figure 1 As shown, the dual-mode defect identification method for substation equipment specifically includes the following steps:

[0028] Step S101, obtaining a first visible light image of transformer oil leakage defects, performing background fusion expansion on the first visible light image based on a preset improved YOLOv8 instance segmentation model, and obtaining a substation equipment structural defect image dataset.

[0029] In this step, see Figure 2 , performing instance segmentation on the first visible light image according to the improved YOLOv8 instance segmentation model to obtain an oil leakage image, and establishing a rectangular coordinate system in the oil leakage image;

[0030] Solve the convex hull of the oil leakage image and divide the convex hull into Graphics, calculation The coordinates of the center of gravity of the figure and area , and calculate the centroid coordinates of the convex hull , the expression is:

[0031] ,

[0032] ,

[0033] A rectangular coordinate system is established in the transformer image without oil leakage defects. The Canny operator is used to detect the edge of the ground where the transformer is located to obtain the ground edge position information. The area surrounded by the edge is defined as the ground area D in the form of a set of coordinate points. A centroid point is selected every preset number of pixels in the transformer image. , each centroid point is screened according to the preset constraint conditions to obtain m centroid points, and the expression of the constraint conditions is:

[0034] ,

[0035] ,

[0036] ,

[0037] wherein, and are respectively the abscissa and ordinate of the points on the convex hull of the oil leakage pattern with as the centroid, is the abscissa of the points on the target detection frame of the transformer;

[0038] For each transformer image without oil leakage defects, only the first best centroid point and the second best centroid point are retained for image augmentation. Among them, the expression of the selection rule for the best centroid point is:

[0039] ,

[0040] wherein, is the distance between any two points among the m centroid points, and are respectively the abscissa and ordinate of the first best centroid point, and are respectively the abscissa and ordinate of the second best centroid point;

[0041] Using the first best centroid point and the second best centroid point as the centroid of the oil leakage image, a new oil leakage defect image is generated in the transformer image without oil leakage defects, and the new oil leakage defect image is added to the structural defect image dataset of substation equipment.

[0042] Step S102, obtain the second visible light image and the infrared image of the substation equipment heating defect, perform registration and fusion on the second visible light image and the infrared image to obtain a fused image dataset.

[0043] In this step, the LOG operator is used to detect the edges of the second visible light image and the infrared image, obtaining a first edge map corresponding to the second visible light image and a second edge map corresponding to the infrared image;

[0044] By traversing the point sets in the first edge map and the second edge map, n contours of the power equipment are searched, and image feature points are obtained using the Harris corner detector in the contours. For each image feature point, the nearest point with local minimum curvature within the contour is searched and defined as the left auxiliary feature point and the right auxiliary feature point. The expression is:

[0045] ,

[0046] ,

[0047] ,

[0048] wherein, is the left auxiliary feature point, is the right auxiliary feature point, is the th image feature point obtained by using the Harris corner detector, is the (f - 1)th image feature point obtained by using the Harris corner detector, is the fth image feature point obtained by using the Harris corner detector, is the (f + 1)th image feature point obtained by using the Harris corner detector, is the th image feature point obtained by using the Harris corner detector, is the domain sampling length, is the normalization constant, is the coordinate of the right auxiliary feature point, is the coordinate of the left auxiliary feature point, is the bilateral filtering parameter, is the pixel point 's coordinate, is the pixel point 's coordinate, is the pixel value of the pixel point , is the pixel value of the pixel point , , are both the standard deviations of the Gaussian function;

[0049] Based on the image feature points, the left auxiliary feature point and the right auxiliary feature point, a feature triangle is formed. A perpendicular line to the opposite side is drawn with the image feature point as the starting point, and the direction of the perpendicular line is defined as the main direction of the feature point;

[0050] The feature points in the second visible light image and the infrared image whose deviation from the main direction is within are considered as a pair of matching points, and the lines are connected between each pair of matching points, and the angles of all the lines relative to the horizontal direction are statistically calculated;

[0051] The angles of all the lines relative to the horizontal direction are divided into intervals, the midpoint of the interval with the largest number of lines and its adjacent intervals is taken as the target angle, the extreme value is determined according to the parabolic method, and the target angle is used as the optimal angle. The matching points deviating from the optimal angle are defined as mismatched points, and the SIFT feature descriptor is used for feature point matching for the mismatched points.

[0052] Specifically, for the image fusion method based on latent low-rank representation, the source images are decomposed into a low-rank part and a salient part. A weighted average strategy is adopted to fuse the low-rank parts to retain more contour information, and the saliency parts are fused by a summation strategy. Then, the fused low-rank part and the fused saliency part are combined to obtain the fused image.

[0053] Step S103: Iteratively train a preset improved YOLOv8 object detection model according to the substation equipment structural defect image dataset and the fused image dataset to obtain a bimodal defect recognition model.

[0054] In this step, corresponding labels are made for each type of defect and the equipment type with defects. Among them, the transformer oil leakage defect is labeled as "sly", the infrared heating defect of substation equipment is labeled as "Defect", the transformer bushing and arrester are labeled as "Transformer Bushing" and "Arrester" respectively. Then, the dataset is divided into a training set, a validation set, and a test set according to the ratio of 8:1:1. An improved YOLOv8 object detection model is built, and then the fused image dataset and the visible light image dataset of transformer oil leakage defects are used to train the improved YOLOv8 object detection model.

[0055] Both the improved YOLOv8 instance segmentation model and the improved YOLOv8 object detection model include a feature extraction network, a feature fusion network, and a detection network.

[0056] The structure of the feature extraction network successively includes: a convolution module, a first C2f module, a first LWA module, a second C2f module, a second LWA module, a third C2f module, a third LWA module, a fourth C2f module, and an SPPF module;

[0057] The structure of the feature fusion network successively includes: a first upsampling module, a first splicing module, a fifth C2f module, a second upsampling module, a second splicing module, a sixth C2f module, a fourth LWA module, a third splicing module, a seventh C2f module, a fifth LWA module, a fourth splicing module, and an eighth C2f module;

[0058] The detection network includes a Detect module;

[0059] Among them, the structures of the first LWA module, the second LWA module, the third LWA module, the fourth LWA module, and the fifth LWA module are exactly the same. The specific operation process is as follows: The image features entering the first LWA module, the second LWA module, the third LWA module, the fourth LWA module, or the fifth LWA module are first divided into two parts. One part extracts local features through an average pooling layer, then is processed by a 1×1 pointwise convolution module, then is processed by an activation layer, and then is input into a multiplication module for array multiplication after being processed together with the other part of the image features. After that, it is summed by a summation module and then output. Among them, the processing process of the other part of the image features is a grouped convolution operation with a convolution kernel of 3.

[0060] Specifically, the output of the second C2f module in the feature extraction network and the output of the second upsampling module in the feature fusion network are connected through the second splicing module; the output of the third C2f module in the feature extraction network and the output of the first upsampling module in the feature fusion network are connected through the first splicing module; the output of the SPPF module in the feature extraction network and the output of the fifth LWA module in the feature fusion network are connected through the fourth splicing module; the output of the fifth C2f module in the feature fusion network and the output of the fourth LWA module in the feature fusion network are connected through the third splicing module.

[0061] It should be noted that the structures of the first C2f module, the second C2f module, the third C2f module, the fourth C2f module, the fifth C2f module, the sixth C2f module, the seventh C2f module, and the eighth C2f module are exactly the same. The specific operation process is as follows: The input first passes through a 1×1 pointwise convolution, and then is split into two parts. One part is directly input into the corresponding splicing module, and the other part is input into the corresponding splicing module after being processed by multiple bottleneck modules. Each time passing through a bottleneck module, an additional branch is generated and input into the corresponding splicing module. Finally, the above parts are input into the splicing module for feature fusion, and then processed by a 1×1 pointwise convolution.

[0062] In a specific embodiment, for the oil leakage defect of the transformer, the defect detection accuracy of the image model for the test set before expanding the dataset of the present application is 59.2%, and the defect detection accuracy after expanding the dataset is increased to 72.3%, and the model accuracy is increased by 13.1%, which fully demonstrates the effectiveness of the method for expanding the dataset of the present invention in improving the model accuracy; for the heating defect of the transformer bushing and arrester, the comparison experimental results before and after the model improvement are shown in Table 1. It can be seen from the results that the defect detection accuracy of the test set images in this embodiment after the model improvement is greatly improved compared with the original model, which demonstrates the effectiveness of the model improvement method of the present application.

[0063] Table 1:

[0064] 。

[0065] In step S104, obtain the first real-time visible light image of the transformer and the real-time fusion image of the substation equipment, input the first real-time visible light image and the real-time fusion image into the bimodal defect recognition model, and the bimodal defect recognition model outputs the oil leakage defect result corresponding to the first real-time visible light image and the heating defect result of the real-time fusion image, where the real-time fusion image is obtained by registering and fusing the second real-time visible light image of the heating defect of the substation equipment and the real-time infrared image.

[0066] In step S105, map the heating defect result to the second real-time visible light image according to the mapping method to obtain the position of the infrared heating defect of the substation equipment.

[0067] In this step, after completing the defect detection of the fusion image of the heating defect of the substation equipment, obtain the position information of the detection frame of the heating defect and its corresponding detection frame of the substation equipment, create an array, store the equipment or defect type in the detection frame in the first column, store the coordinate positions of the 4 endpoints of the detection frame in the second to fifth columns, store the confidence of the object category in the frame in the sixth column, and use the cv2 library in python to draw the equipment and defect detection frames in the visible light image corresponding to the fusion image of the heating defect of the substation equipment according to the data in the array, so that the operation and maintenance personnel can intuitively identify the specific position of the infrared heating defect of the equipment in the visible light inspection image. For the heating defects of transformer bushings and arresters, the defect detection results before and after mapping are as Figure 3 shown.

[0068] In summary, the method of this application collects visible light image samples of transformer oil leakage defects and visible light and infrared bimodal image samples of substation equipment heating defects; in view of the insufficient number of transformer oil leakage defect samples, an improved YOLOv8 instance segmentation model is built, and the defect samples are expanded by using the method of image segmentation and background fusion; a fusion image dataset is constructed through image registration and fusion, an improved YOLOv8 object detection model is built, the transformer oil leakage defect samples and fusion image samples are used for the training of the improved YOLOv8 object detection model, and then the trained improved YOLOv8 object detection model is used to identify the transformer oil leakage defects and substation equipment heating defects; in view of the problem that the heating defect detection results of substation equipment are difficult to visually present in the operation and maintenance work, the mapping method is used to present the heating defect detection results of the fusion image in the visible light image, and by introducing the deep learning method into the field of substation equipment operation and maintenance technology, the efficient, accurate and intelligent identification of the structural defects of substation equipment is realized.

[0069] Please refer to Figure 4, which shows a structural block diagram of a dual - mode defect recognition system for substation equipment of the present application.

[0070] As Figure 4 shown, the dual - mode defect recognition system 200 for substation equipment includes an expansion module 210, a fusion module 220, a training module 230, an output module 240, and a mapping module 250.

[0071] Among them, the expansion module 210 is configured to obtain a first visible - light image of the transformer oil leakage defect, and perform background fusion and expansion on the first visible - light image based on a preset improved YOLOv8 instance segmentation model to obtain a substation equipment structural defect image dataset; the fusion module 220 is configured to obtain a second visible - light image and an infrared image of the substation equipment heating defect, perform registration and fusion on the second visible - light image and the infrared image to obtain a fusion image dataset; the training module 230 is configured to perform iterative training on a preset improved YOLOv8 object detection model according to the substation equipment structural defect image dataset and the fusion image dataset to obtain a dual - mode defect recognition model; the output module 240 is configured to obtain a first real - time visible - light image of the transformer and a real - time fusion image of the substation equipment, input the first real - time visible - light image and the real - time fusion image into the dual - mode defect recognition model, and the dual - mode defect recognition model outputs an oil leakage defect result corresponding to the first real - time visible - light image and a heating defect result of the real - time fusion image, where the real - time fusion image is obtained by registering and fusing a second real - time visible - light image of the substation equipment heating defect and a real - time infrared image; the mapping module 250 is configured to map the heating defect result to the second real - time visible - light image according to the mapping method to obtain the position of the substation equipment infrared heating defect.

[0072] It should be understood that Figure 4 the modules described in Figure 1 correspond to the respective steps in the method described with reference to Figure 4 . Thus, the operations, features, and corresponding technical effects described above for the method also apply to

[0073] the modules in

[0074] In some other embodiments, the embodiment of the present invention further provides a computer - readable storage medium, on which a computer program is stored. When the program instructions are executed by a processor, the processor is caused to execute the substation equipment dual - mode defect recognition method in any of the above - mentioned method embodiments;

[0075] Obtain the first visible light image of the transformer oil leakage defect, and perform background fusion and augmentation on the first visible light image based on a preset improved YOLOv8 instance segmentation model to obtain a structural defect image dataset of substation equipment;

[0076] Obtain the second visible light image and the infrared image of the substation equipment heating defect, register and fuse the second visible light image and the infrared image to obtain a fused image dataset;

[0077] Iteratively train a preset improved YOLOv8 object detection model according to the structural defect image dataset of the substation equipment and the fused image dataset to obtain a bimodal defect recognition model;

[0078] Obtain the first real-time visible light image of the transformer and the real-time fused image of the substation equipment, and input the first real-time visible light image and the real-time fused image into the bimodal defect recognition model. The bimodal defect recognition model outputs the oil leakage defect result corresponding to the first real-time visible light image and the heating defect result corresponding to the real-time fused image, where the real-time fused image is obtained by registering and fusing the second real-time visible light image of the substation equipment heating defect and the real-time infrared image;

[0079] Map the heating defect result to the second real-time visible light image according to the mapping method to obtain the position of the infrared heating defect of the substation equipment.

[0080] A computer-readable storage medium may include a storage program area and a storage data area. Among them, the storage program area can store an operating system and application programs required for at least one function; the storage data area can store data created according to the use of the substation equipment bimodal defect recognition system, etc. In addition, the computer-readable storage medium may include a high-speed random access memory, and may also include a memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the computer-readable storage medium may optionally include a memory remotely set relative to the processor, and these remote memories can be connected to the substation equipment bimodal defect recognition system through a network. Examples of the above networks include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0081] Figure 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention, as Figure 5 shown, the device includes: a processor 310 and a memory 320. The electronic device may further include: an input device 330 and an output device 340. The processor 310, the memory 320, the input device 330, and the output device 340 may be connected through a bus or other means, Figure 5Take the bus connection as an example. The memory 320 is the above-mentioned computer-readable storage medium. The processor 310 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions, and modules stored in the memory 320, that is, implements the dual-modal defect recognition method for substation equipment in the above method embodiment. The input device 330 can receive input digital or character information, and generate key signal inputs related to user settings and function control of the dual-modal defect recognition system for substation equipment. The output device 340 can include display devices such as a display screen.

[0082] The above electronic device can execute the method provided by the embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the method. For technical details not described in detail in this embodiment, reference can be made to the method provided by the embodiment of the present invention.

[0083] As an implementation manner, the above electronic device is applied to a dual-modal defect recognition system for substation equipment and is used for a client, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to:

[0084] Obtain a first visible light image of the transformer oil leakage defect, and perform background fusion and augmentation on the first visible light image based on a preset improved YOLOv8 instance segmentation model to obtain a substation equipment structural defect image dataset;

[0085] Obtain a second visible light image and an infrared image of the substation equipment heating defect, perform registration and fusion on the second visible light image and the infrared image to obtain a fusion image dataset;

[0086] Iteratively train a preset improved YOLOv8 object detection model according to the substation equipment structural defect image dataset and the fusion image dataset to obtain a dual-modal defect recognition model;

[0087] Obtain a first real-time visible light image of the transformer and a real-time fusion image of the substation equipment, input the first real-time visible light image and the real-time fusion image into the dual-modal defect recognition model, and the dual-modal defect recognition model outputs an oil leakage defect result corresponding to the first real-time visible light image and a heating defect result corresponding to the real-time fusion image, wherein the real-time fusion image is obtained by registering and fusing a second real-time visible light image of the substation equipment heating defect and a real-time infrared image;

[0088] Map the heating defect result to the second real-time visible light image according to the mapping method to obtain the position of the substation equipment infrared heating defect.

[0089] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiments.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present invention.

Claims

1. A dual-modal defect recognition method for substation equipment, characterized in that, Including: Obtain the first visible light image of the transformer oil leakage defect, and perform background fusion and augmentation on the first visible light image based on a preset improved YOLOv8 instance segmentation model to obtain a structural defect image dataset of substation equipment. The process of performing background fusion and augmentation on the first visible light image based on the preset improved YOLOv8 instance segmentation model to obtain a structural defect image dataset of substation equipment includes: Obtain the oil leakage image according to the instance segmentation of the first visible light image by the improved YOLOv8 instance segmentation model, and establish a rectangular coordinate system in the oil leakage image; Solve the convex hull of the oil leakage image and divide the convex hull into Graphics, calculation The coordinates of the center of gravity of the figure and area , and calculate the centroid coordinates of the convex hull , the expression is: ， ， Establish a rectangular coordinate system in the transformer image without oil leakage defects, use the Canny operator to detect the edge of the ground where the transformer is located, obtain the ground edge position information, and define the area surrounded by the edge as the ground area D in the form of a set of coordinate points. Select a center of gravity point every preset number of pixels in the transformer image , and screen each center of gravity point according to the preset constraint conditions to obtain m center of gravity points. The expression of the constraint conditions is: ， ， ， In the formula, , are respectively the abscissa and ordinate of the points on the convex hull of the oil leakage pattern with as the centroid, is the abscissa of the point on the target detection frame of the transformer; For each transformer image without oil leakage defects, only the first best center of gravity point and the second best center of gravity point are used for image expansion. Among them, the expression of the selection rule for the best center of gravity point is: ， In the formula, is the distance between any two of the m center-of-gravity points, , are respectively the abscissa and ordinate of the first optimal center-of-gravity point, , are respectively the abscissa and ordinate of the second optimal center-of-gravity point; With the first optimal center of gravity and the second optimal center of gravity As the center of gravity of the oil leakage image, a new oil leakage defect image is generated in the transformer image without oil leakage defects, and the new oil leakage defect image is added to the structural defect image dataset of substation equipment; Obtain the second visible light image and the infrared image of the substation equipment heating defect, and perform registration and fusion on the second visible light image and the infrared image to obtain a fusion image dataset; Iteratively train a preset improved YOLOv8 object detection model according to the structural defect image dataset of substation equipment and the fusion image dataset to obtain a bimodal defect recognition model. Among them, both the improved YOLOv8 instance segmentation model and the improved YOLOv8 object detection model include a feature extraction network, a feature fusion network, and a detection network; The structure of the feature extraction network sequentially includes: a convolution module, a first C2f module, a first LWA module, a second C2f module, a second LWA module, a third C2f module, a third LWA module, a fourth C2f module, and an SPPF module; The structure of the feature fusion network sequentially includes: a first upsampling module, a first splicing module, a fifth C2f module, a second upsampling module, a second splicing module, a sixth C2f module, a fourth LWA module, a third splicing module, a seventh C2f module, a fifth LWA module, a fourth splicing module, and an eighth C2f module; The detection network includes a Detect module; Among them, the structures of the first LWA module, the second LWA module, the third LWA module, the fourth LWA module, and the fifth LWA module are exactly the same. The specific operation process is as follows: The image features entering the first LWA module, the second LWA module, the third LWA module, the fourth LWA module, or the fifth LWA module are first divided into two parts. One part extracts local features through an average pooling layer, then is processed by a 1×1 pointwise convolution module, then is processed by an activation layer, and then is input into a multiplication module for array multiplication with the other part of the processed image features. After that, it is summed by a summation module and then output. Among them, the processing process of the other part of the image features is a grouped convolution operation with a convolution kernel of 3. Obtain the first real-time visible light image of the transformer and the real-time fusion image of the substation equipment, input the first real-time visible light image and the real-time fusion image into the dual-modal defect recognition model, and the dual-modal defect recognition model outputs the leakage oil defect result corresponding to the first real-time visible light image and the heating defect result of the real-time fusion image. Among them, the real-time fusion image is obtained by registering and fusing the second real-time visible light image of the substation equipment heating defect and the real-time infrared image; According to the mapping method, map the heating defect result to the second real-time visible light image to obtain the position of the infrared heating defect of the substation equipment.

2. A dual-modal defect recognition method for substation equipment according to claim 1, characterized in that, Among them, The output of the second C2f module in the feature extraction network is connected to the output of the second upsampling module in the feature fusion network through the second splicing module; The output of the third C2f module in the feature extraction network is connected to the output of the first upsampling module in the feature fusion network through the first splicing module; The output of the SPPF module in the feature extraction network is connected to the output of the fifth LWA module in the feature fusion network through the fourth splicing module; The output of the fifth C2f module in the feature fusion network is connected to the output of the fourth LWA module in the feature fusion network through the third splicing module.

3. A bimodal defect recognition system for substation equipment, characterized in that, Include: An expansion module configured to obtain the first visible light image of the transformer leakage oil defect, and perform background fusion expansion on the first visible light image based on a preset improved YOLOv8 instance segmentation model to obtain a substation equipment structural defect image dataset. The performing background fusion expansion on the first visible light image based on a preset improved YOLOv8 instance segmentation model to obtain a substation equipment structural defect image dataset includes: Perform instance segmentation on the first visible light image according to the improved YOLOv8 instance segmentation model to obtain a leakage oil image, and establish a rectangular coordinate system in the leakage oil image; Solve the convex hull of the oil leakage image and divide the convex hull into Graphics, calculation The coordinates of the center of gravity of the figure and area , and calculate the centroid coordinates of the convex hull , the expression is: ， ， Establish a rectangular coordinate system in the transformer image without oil leakage defects. Use the Canny operator to detect the edge of the ground where the transformer is located, obtain the ground edge position information, and define the area surrounded by the edge as the ground area D in the form of a set of coordinate points. Select a centroid point every preset number of pixels in the transformer image , and screen each centroid point according to the preset constraint conditions to obtain m centroid points. The expression of the constraint conditions is as follows: ， ， ， Wherein, , are respectively the abscissa and ordinate of the points on the convex hull of the oil leakage pattern with as the center of gravity, is the abscissa of the point on the target detection frame of the transformer; For each transformer image without oil leakage defects, only the first best center of gravity point and the second best center of gravity point are used for image expansion. Among them, the expression of the selection rule for the best center of gravity point is: ， In the formula, is the distance between any two of the m center-of-gravity points, and are the abscissa and ordinate of the first optimal center-of-gravity point respectively, and are the abscissa and ordinate of the second optimal center-of-gravity point respectively; Using the first optimal center of gravity point and the second optimal center of gravity point as the center of gravity of the oil leakage image, a new oil leakage defect image is generated in the transformer image without oil leakage defects, and the new oil leakage defect image is added to the structural defect image dataset of substation equipment; A fusion module configured to obtain the second visible light image and the infrared image of the substation equipment heating defect, and perform registration and fusion on the second visible light image and the infrared image to obtain a fusion image dataset; A training module configured to perform iterative training on a preset improved YOLOv8 object detection model according to the substation equipment structural defect image dataset and the fusion image dataset to obtain a dual-modal defect recognition model. Among them, both the improved YOLOv8 instance segmentation model and the improved YOLOv8 object detection model include a feature extraction network, a feature fusion network, and a detection network; The structure of the feature extraction network sequentially includes: a convolution module, a first C2f module, a first LWA module, a second C2f module, a second LWA module, a third C2f module, a third LWA module, a fourth C2f module, and an SPPF module; The structure of the feature fusion network successively includes: a first upsampling module, a first splicing module, a fifth C2f module, a second upsampling module, a second splicing module, a sixth C2f module, a fourth LWA module, a third splicing module, a seventh C2f module, a fifth LWA module, a fourth splicing module, and an eighth C2f module; The detection network includes a Detect module; Among them, the structures of the first LWA module, the second LWA module, the third LWA module, the fourth LWA module, and the fifth LWA module are exactly the same. The specific operation process is as follows: The image features entering the first LWA module, the second LWA module, the third LWA module, the fourth LWA module, or the fifth LWA module are first divided into two parts. One part extracts local features through an average pooling layer, then is processed by a 1×1 pointwise convolution module, then is processed by an activation layer, and then is input into a multiplication module for array multiplication after being processed together with the other part of the image features. After that, the sum is obtained through a summation module and then output. Among them, the processing process of the other part of the image features is a grouped convolution operation with a convolution kernel of 3; An output module, configured to obtain a first real-time visible light image of the transformer and a real-time fusion image of the substation equipment, input the first real-time visible light image and the real-time fusion image into the bimodal defect recognition model, and the bimodal defect recognition model outputs an oil leakage defect result corresponding to the first real-time visible light image and a heating defect result corresponding to the real-time fusion image. Among them, the real-time fusion image is obtained by registering and fusing a second real-time visible light image of the substation equipment with a heating defect and a real-time infrared image; A mapping module, configured to map the heating defect result to the second real-time visible light image according to the mapping method to obtain the position of the infrared heating defect of the substation equipment.

4. An electronic device, characterized in that, Includes: At least one processor, and a memory communicatively connected to the at least one processor. Among them, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1 to 2.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method according to any one of claims 1 to 2.

Images