Method and system for identifying bimodal defects of power transformation equipment

Abstract

The invention discloses a power transformation equipment bimodal defect identification method and system. The method comprises the following steps: collecting a transformer oil leakage defect visible light image sample and a power transformation equipment heating defect visible light and infrared bimodal image sample; building an improved YOLOv8 instance segmentation model, and expanding a defect sample by adopting an image segmentation and background fusion method; the method comprises the following steps of: constructing a fusion image data set through image registration and fusion, constructing an improved YOLOv8 target detection model, using a transformer oil leakage defect sample and a fusion image sample for training the improved YOLOv8 target detection model, and identifying a transformer oil leakage defect and a power transformation equipment heating defect through the trained improved YOLOv8 target detection model; and presenting a heating defect detection result of the fused image in the visible light image by adopting a mapping method. By introducing a deep learning method into the technical field of operation and maintenance of the power transformation equipment, efficient, accurate and intelligent recognition of structural defects of the power transformation equipment is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of power transmission and transformation equipment operation and maintenance, and in particular relates to a dual-mode defect identification method and system for power transformation equipment. Background Art

[0002] The power equipment in the substation is the core component of the substation, playing the role of voltage conversion, power distribution and power protection. However, as time goes by, the substation equipment is prone to various structural defects and heating defects. Common typical defect types include transformer oil leakage defects, transformer bushing heating defects and lightning arrester heating defects. The current operation and maintenance of substation equipment mainly relies on manual inspections and regular maintenance, which is not only labor-intensive and inefficient, but also difficult to achieve real-time monitoring. Summary of the invention

[0003] The present invention provides a dual-mode defect identification method and system for substation equipment, which are used to solve the technical problem that the operation and maintenance of substation equipment mainly depends on manual inspection and regular maintenance.

[0004] In a first aspect, the present invention provides a dual-mode defect identification method for substation equipment, comprising: Acquire a first visible light image of the transformer oil leakage defect, perform background fusion expansion on the first visible light image based on a preset improved YOLOv8 instance segmentation model, and obtain a substation equipment structural defect image dataset; Acquire a second visible light image and an infrared image of a heating defect of a power transformation equipment, and register and fuse the second visible light image and the infrared image to obtain a fused image data set; Iteratively training a preset improved YOLOv8 target detection model according to the substation equipment structural defect image dataset and the fused image dataset to obtain a dual-modal defect recognition model; Acquire a first real-time visible light image of the transformer and a real-time fused image of the transformer substation equipment, input the first real-time visible light image and the real-time fused image into the dual-modal defect recognition model, the dual-modal defect recognition model outputs an oil leakage defect result corresponding to the first real-time visible light image, and a heating defect result corresponding to the real-time fused image, wherein the real-time fused image is obtained by registering and fusing a second real-time visible light image of the transformer substation heating defect and a real-time infrared image; The heating defect result is mapped to the second real-time visible light image according to a mapping method to obtain the location of the infrared heating defect of the substation equipment.

[0005] In a second aspect, the present invention provides a dual-mode defect identification system for substation equipment, comprising: An expansion module is configured to obtain a first visible light image of a transformer oil leakage defect, and perform background fusion expansion on the first visible light image based on a preset improved YOLOv8 instance segmentation model to obtain a substation equipment structural defect image dataset; a fusion module configured to obtain a second visible light image and an infrared image of a heating defect of a substation equipment, register and fuse the second visible light image and the infrared image, and obtain a fused image data set; A training module, configured to iteratively train a preset improved YOLOv8 target detection model according to the substation equipment structural defect image dataset and the fused image dataset to obtain a dual-modal defect recognition model; an output module, configured to obtain a first real-time visible light image of the transformer and a real-time fused image of the transformer substation, input the first real-time visible light image and the real-time fused image into the dual-modal defect recognition model, and the dual-modal defect recognition model outputs an oil leakage defect result corresponding to the first real-time visible light image and a heating defect result corresponding to the real-time fused image, wherein the real-time fused image is obtained by registering and fusion of a second real-time visible light image of the transformer substation heating defect and a real-time infrared image; A mapping module is configured to map the heating defect result to the second real-time visible light image according to a mapping method to obtain the location of the infrared heating defect of the substation equipment.

[0006] According to a third aspect, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the dual-modal defect identification method for substation equipment of any embodiment of the present invention.

[0007] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein when the program instructions are executed by a processor, the processor executes the steps of the dual-modal defect identification method for substation equipment of any embodiment of the present invention.

[0008] The dual-modal defect recognition method and system of substation equipment of the present application collect visible light image samples of transformer oil leakage defects and visible light and infrared dual-modal image samples of substation heating defects; in view of the insufficient number of transformer oil leakage defect samples, an improved YOLOv8 instance segmentation model is built, and the defect samples are expanded by using the image segmentation and background fusion method; a fused image data set is constructed by image registration and fusion, an improved YOLOv8 target detection model is built, and the transformer oil leakage defect samples and the fused image samples are used for training the improved YOLOv8 target detection model, and then the transformer oil leakage defect and substation heating defect are identified by the trained improved YOLOv8 target detection model; in view of the problem that the detection results of substation heating defects are difficult to be intuitively presented in the operation and maintenance work, a mapping method is used to present the heating defect detection results of the fused image in the visible light image, and by introducing the deep learning method into the field of substation operation and maintenance technology, efficient, accurate and intelligent recognition of structural defects of substation equipment is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A flowchart of a dual-mode defect identification method for substation equipment provided by an embodiment of the present invention; Figure 2 A flowchart of a method for using multi-intent recognition provided by an embodiment of the present invention; Figure 3 A flow chart of a training phase of a specific embodiment is provided for an embodiment of the present invention; Figure 4 A structural block diagram of a dual-mode defect identification system for substation equipment provided by an embodiment of the present invention; Figure 5 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0011] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0012] See also Figure 1 , which shows a flow chart of a dual-modal defect identification method for substation equipment of the present application.

[0013] like Figure 1 As shown, the dual-mode defect identification method for substation equipment specifically includes the following steps: Step S101, obtaining a first visible light image of transformer oil leakage defects, performing background fusion expansion on the first visible light image based on a preset improved YOLOv8 instance segmentation model, and obtaining a substation equipment structural defect image dataset.

[0014] In this step, see Figure 2 , performing instance segmentation on the first visible light image according to the improved YOLOv8 instance segmentation model to obtain an oil leakage image, and establishing a rectangular coordinate system in the oil leakage image; Solve the convex hull of the oil leakage image and divide the convex hull into Graphics, calculation The coordinates of the center of gravity of the figure and area , and calculate the centroid coordinates of the convex hull , the expression is: , , A rectangular coordinate system is established in the transformer image without oil leakage defects. The Canny operator is used to detect the edge of the ground where the transformer is located to obtain the ground edge position information. The area surrounded by the edge is defined as the ground area D in the form of a set of coordinate points. A centroid point is selected every preset number of pixels in the transformer image. , each centroid point is screened according to the preset constraint conditions to obtain m centroid points, and the expression of the constraint conditions is: , , , In the formula, , For The horizontal and vertical coordinates of the points on the convex hull of the oil leakage graph are the centroids, is the horizontal coordinate of the point on the transformer target detection box; For each transformer image without oil leakage defect, only the first best center of gravity point is retained. and the second best center of gravity For image expansion, the expression of the best center of gravity selection rule is: , In the formula, is the distance between any two points among the m centroids, , are the horizontal and vertical coordinates of the first optimal center of gravity, , are the horizontal and vertical coordinates of the second best center of gravity respectively; First best center of gravity and the second best center of gravity A new oil leakage defect image is generated from the transformer image without oil leakage defect as the center of gravity of the oil leakage image, and the new oil leakage defect image is added to the structural defect image dataset of substation equipment.

[0015] Step S102: acquiring a second visible light image and an infrared image of a heating defect of a power transformation equipment, registering and fusing the second visible light image and the infrared image to obtain a fused image data set.

[0016] In this step, the LOG operator is used to detect the edges of the second visible light image and the infrared image, and a first edge map corresponding to the second visible light image and a second edge map corresponding to the infrared image are obtained; By traversing the point sets in the first edge map and the second edge map, n contours of the power equipment are obtained. Harris corner detector is used to obtain image feature points in the contours. For each image feature point, the nearest point with local minimum curvature in the contour is searched, which is defined as the left auxiliary feature point and the right auxiliary feature point. The expression is: , , , In the formula, is the left auxiliary feature point, is the right auxiliary feature point, is the first corner point obtained by using Harris corner detector image feature points, is the f-1th image feature point obtained using Harris corner detector, is the f-th image feature point obtained using Harris corner detector, is the f+1th image feature point obtained using Harris corner detector, is the first corner point obtained by using Harris corner detector image feature points, is the field sampling length, is the normalization constant, is the coordinate of the right auxiliary feature point, is the coordinate of the left auxiliary feature point, is the bilateral filter parameter, Pixel The coordinates of Pixel The coordinates of Pixel The pixel value of Pixel The pixel value of , are the standard deviations of Gaussian functions; A feature triangle is formed based on the image feature point, the left auxiliary feature point and the right auxiliary feature point, a perpendicular line is drawn to the opposite side with the image feature point as the starting point, and the direction of the perpendicular line is defined as the main direction of the feature point; The second visible light image and the infrared image are compared for the main direction deviation The feature points within are considered as a pair of matching points, and lines are drawn between the matching points, and the angles of all lines relative to the horizontal direction are counted; Divide the angles of all lines relative to the horizontal into intervals, take the target angle corresponding to the midpoint of the interval with the largest number of connecting lines and its adjacent intervals, determine the extreme value according to the parabola method and use the target angle as the optimal angle, and deviate from the optimal angle The matching points are defined as mismatched points, and the SIFT feature descriptor is used to perform feature point matching for the mismatched points.

[0017] Specifically, the image fusion method based on latent low-rank representation decomposes the source image into a low-rank part and a salient part, adopts a weighted average strategy to fuse the low-rank part to retain more contour information, and fuses the salient part through a summation strategy, and then combines the fused low-rank part and the fused salient part to obtain a fused image.

[0018] Step S103, iteratively training a preset improved YOLOv8 target detection model according to the substation equipment structural defect image dataset and the fused image dataset to obtain a dual-modal defect recognition model.

[0019] In this step, corresponding labels are made for each type of defect and the type of equipment where the defect occurs. The transformer oil leakage defect is labeled as "sly", the substation equipment infrared heating defect is labeled as "Defect", the transformer bushing and the arrester are labeled as "Transformer Bushing" and "Arrester", respectively. Then, the dataset is divided into training set, validation set and test set in a ratio of 8:1:1. An improved YOLOv8 target detection model is built, and then the fused image dataset and the transformer oil leakage defect visible light image dataset are used to train the improved YOLOv8 target detection model.

[0020] The improved YOLOv8 instance segmentation model and the improved YOLOv8 target detection model both include a feature extraction network, a feature fusion network, and a detection network.

[0021] The structure of the feature extraction network includes: a convolution module, a first C2f module, a first LWA module, a second C2f module, a second LWA module, a third C2f module, a third LWA module, a fourth C2f module and an SPPF module in sequence; The structure of the feature fusion network includes, in sequence: a first upsampling module, a first splicing module, a fifth C2f module, a second upsampling module, a second splicing module, a sixth C2f module, a fourth LWA module, a third splicing module, a seventh C2f module, a fifth LWA module, a fourth splicing module, and an eighth C2f module; The detection network includes the Detect module; Among them, the structures of the first LWA module, the second LWA module, the third LWA module, the fourth LWA module and the fifth LWA module are exactly the same. The specific operation process is: the image features entering the first LWA module, the second LWA module, the third LWA module, the fourth LWA module or the fifth LWA module are first divided into two parts, one of which is used to extract local features through the average pooling layer, and then processed by a 1×1 point-by-point convolution module, and then processed by an activation layer, and then input into the multiplication module after the processing with the other image features for array multiplication, and then output after summing through the summation module, wherein the processing process of the other image features is a grouped convolution operation with a convolution kernel of 3.

[0022] Specifically, the output of the second C2f module in the feature extraction network is connected to the output of the second upsampling module in the feature fusion network through the second splicing module; the output of the third C2f module in the feature extraction network is connected to the output of the first upsampling module in the feature fusion network through the first splicing module; the output of the SPPF module in the feature extraction network is connected to the output of the fifth LWA module in the feature fusion network through the fourth splicing module; the output of the fifth C2f module in the feature fusion network is connected to the output of the fourth LWA module in the feature fusion network through the third splicing module.

[0023] It should be noted that the structures of the first C2f module, the second C2f module, the third C2f module, the fourth C2f module, the fifth C2f module, the sixth C2f module, the seventh C2f module and the eighth C2f module are exactly the same. The specific operation process is: the input first undergoes a 1×1 point-by-point convolution and then is divided into two parts, one of which is directly input into the corresponding splicing module, and the other is input into the corresponding splicing module after being processed by multiple bottleneck modules. Each time it passes through a bottleneck module, an additional branch will be generated and input into the corresponding splicing module. Finally, the above parts are input into the splicing module for feature fusion, and then processed by a 1×1 point-by-point convolution.

[0024] In a specific embodiment, for transformer oil leakage defects, the defect detection accuracy of the test set image model before the data set expansion of the present application was 59.2%, and the defect detection accuracy after the data set expansion was improved to 72.3%, and the model accuracy was improved by 13.1%, which fully demonstrated the effectiveness of the data set expansion method of the present invention in improving the model accuracy; for the heating defects of transformer bushings and lightning arresters, the comparative experimental results before and after the model improvement are shown in Table 1. From the results, it can be seen that the defect detection accuracy of the test set images in this embodiment after the model improvement is greatly improved compared with the original model, which illustrates the effectiveness of the model improvement method of the present application.

[0025] Table 1: .

[0026] Step S104, obtaining a first real-time visible light image of the transformer and a real-time fused image of the substation equipment, inputting the first real-time visible light image and the real-time fused image into the dual-modal defect recognition model, the dual-modal defect recognition model outputting an oil leakage defect result corresponding to the first real-time visible light image, and a heating defect result corresponding to the real-time fused image, wherein the real-time fused image is obtained by registering and fusing a second real-time visible light image of the heating defect of the substation equipment and a real-time infrared image.

[0027] Step S105 , mapping the heating defect result to the second real-time visible light image according to a mapping method, and obtaining the location of the infrared heating defect of the substation equipment.

[0028] In this step, after completing the defect detection of the fusion image of the heating defect of the substation equipment, obtain the detection frame of the heating defect and the position information of the corresponding detection frame of the substation equipment, create an array, the first column stores the equipment or defect type in the detection frame, the second to fifth columns store the coordinate positions of the four endpoints of the detection frame, and the sixth column stores the confidence of the object category in the frame. Use the cv2 library in python to draw the equipment and defect detection frame in the visible light image corresponding to the fusion image of the heating defect of the substation equipment according to the data in the array, so that the operation and maintenance personnel can intuitively identify the specific location of the infrared heating defect of the equipment in the visible light inspection image. For the heating defect of the transformer bushing and the heating defect of the lightning arrester, the defect detection results before and after mapping are as follows: Figure 3 shown.

[0029] In summary, the method of the present application collects visible light image samples of transformer oil leakage defects and visible light and infrared dual-modal image samples of substation heating defects; in view of the insufficient number of transformer oil leakage defect samples, an improved YOLOv8 instance segmentation model is built, and the defect samples are expanded by using the image segmentation and background fusion method; a fused image dataset is constructed through image registration and fusion, an improved YOLOv8 target detection model is built, and the transformer oil leakage defect samples and the fused image samples are used for training the improved YOLOv8 target detection model, and then the transformer oil leakage defect and substation heating defect are identified by the trained improved YOLOv8 target detection model; in view of the problem that the detection results of substation heating defects are difficult to be intuitively presented in the operation and maintenance work, a mapping method is used to present the heating defect detection results of the fused image in the visible light image, and by introducing the deep learning method into the field of substation operation and maintenance technology, efficient, accurate and intelligent identification of structural defects of substation equipment is achieved.

[0030] See also Figure 4 , which shows a structural block diagram of a dual-modal defect identification system for substation equipment of the present application.

[0031] like Figure 4 As shown, the dual-modal defect identification system 200 for substation equipment includes an expansion module 210 , a fusion module 220 , a training module 230 , an output module 240 and a mapping module 250 .

[0032] Among them, the expansion module 210 is configured to obtain a first visible light image of the transformer oil leakage defect, and perform background fusion expansion on the first visible light image based on the preset improved YOLOv8 instance segmentation model to obtain a data set of structural defect images of the substation equipment; the fusion module 220 is configured to obtain a second visible light image and an infrared image of the substation equipment heating defect, and perform registration and fusion on the second visible light image and the infrared image to obtain a fused image data set; the training module 230 is configured to iteratively train the preset improved YOLOv8 target detection model according to the substation equipment structural defect image data set and the fused image data set to obtain a dual-modal defect recognition model; the input The output module 240 is configured to obtain a first real-time visible light image of the transformer and a real-time fused image of the substation equipment, and input the first real-time visible light image and the real-time fused image into the dual-modal defect recognition model, and the dual-modal defect recognition model outputs an oil leakage defect result corresponding to the first real-time visible light image, and a heating defect result corresponding to the real-time fused image, wherein the real-time fused image is obtained by aligning and fusing a second real-time visible light image of the heating defect of the substation equipment and a real-time infrared image; the mapping module 250 is configured to map the heating defect result to the second real-time visible light image according to a mapping method, and obtain the location of the infrared heating defect of the substation equipment.

[0033] It should be understood that Figure 4 Modules and references documented in Figure 1 Therefore, the operations and features described above for the method and the corresponding technical effects are also applicable to Figure 4 The modules in it will not be described in detail here.

[0034] In some other embodiments, the embodiments of the present invention further provide a computer-readable storage medium having a computer program stored thereon, wherein when the program instructions are executed by a processor, the processor is caused to execute the dual-mode defect identification method for substation equipment in any of the above method embodiments; As an implementation mode, the computer-readable storage medium of the present invention stores computer-executable instructions, and the computer-executable instructions are configured as follows: Acquire a first visible light image of the transformer oil leakage defect, perform background fusion expansion on the first visible light image based on a preset improved YOLOv8 instance segmentation model, and obtain a substation equipment structural defect image dataset; Acquire a second visible light image and an infrared image of a heating defect of a power transformation equipment, and register and fuse the second visible light image and the infrared image to obtain a fused image data set; Iteratively training a preset improved YOLOv8 target detection model according to the substation equipment structural defect image dataset and the fused image dataset to obtain a dual-modal defect recognition model; Acquire a first real-time visible light image of the transformer and a real-time fused image of the transformer substation equipment, input the first real-time visible light image and the real-time fused image into the dual-modal defect recognition model, the dual-modal defect recognition model outputs an oil leakage defect result corresponding to the first real-time visible light image, and a heating defect result corresponding to the real-time fused image, wherein the real-time fused image is obtained by registering and fusing a second real-time visible light image of the transformer substation heating defect and a real-time infrared image; The heating defect result is mapped to the second real-time visible light image according to a mapping method to obtain the location of the infrared heating defect of the substation equipment.

[0035] The computer-readable storage medium may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required by at least one function; the data storage area may store data created according to the use of the dual-modal defect identification system for substation equipment, etc. In addition, the computer-readable storage medium may include a high-speed random access memory, and may also include a memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the computer-readable storage medium may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the dual-modal defect identification system for substation equipment via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0036] Figure 5 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Figure 5 As shown, the device includes: a processor 310 and a memory 320. The electronic device may also include: an input device 330 and an output device 340. The processor 310, the memory 320, the input device 330 and the output device 340 may be connected via a bus or other means. Figure 5 In the example, the connection through the bus is taken as an example. The memory 320 is the above-mentioned computer-readable storage medium. The processor 310 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 320, that is, the dual-modal defect identification method of the substation equipment in the above-mentioned method embodiment is realized. The input device 330 can receive input digital or character information, and generate key signal input related to user settings and function control of the dual-modal defect identification system of the substation equipment. The output device 340 may include display devices such as display screens.

[0037] The electronic device can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the method provided by the embodiment of the present invention.

[0038] As an implementation mode, the electronic device is applied to a dual-mode defect identification system for substation equipment, and is used for a client, and includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can: Acquire a first visible light image of the transformer oil leakage defect, perform background fusion expansion on the first visible light image based on a preset improved YOLOv8 instance segmentation model, and obtain a substation equipment structural defect image dataset; Acquire a second visible light image and an infrared image of a heating defect of a power transformation equipment, and register and fuse the second visible light image and the infrared image to obtain a fused image data set; Iteratively training a preset improved YOLOv8 target detection model according to the substation equipment structural defect image dataset and the fused image dataset to obtain a dual-modal defect recognition model; Acquire a first real-time visible light image of the transformer and a real-time fused image of the transformer substation equipment, input the first real-time visible light image and the real-time fused image into the dual-modal defect recognition model, the dual-modal defect recognition model outputs an oil leakage defect result corresponding to the first real-time visible light image, and a heating defect result corresponding to the real-time fused image, wherein the real-time fused image is obtained by registering and fusing a second real-time visible light image of the transformer substation heating defect and a real-time infrared image; The heating defect result is mapped to the second real-time visible light image according to a mapping method to obtain the location of the infrared heating defect of the substation equipment.

[0039] Through the description of the above implementation modes, those skilled in the art can clearly understand that each implementation mode can be implemented by means of software plus a necessary general hardware platform, or of course by hardware. Based on such an understanding, the above technical solution can essentially or in other words be embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0040] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dual-mode defect identification method for substation equipment, characterized in that: include: Acquire a first visible light image of the transformer oil leakage defect, perform background fusion expansion on the first visible light image based on a preset improved YOLOv8 instance segmentation model, and obtain a substation equipment structural defect image dataset; Acquire a second visible light image and an infrared image of a heating defect of a power transformation equipment, and register and fuse the second visible light image and the infrared image to obtain a fused image data set; Iteratively training a preset improved YOLOv8 target detection model according to the substation equipment structural defect image dataset and the fused image dataset to obtain a dual-modal defect recognition model; Acquire a first real-time visible light image of the transformer and a real-time fused image of the transformer substation equipment, input the first real-time visible light image and the real-time fused image into the dual-modal defect recognition model, the dual-modal defect recognition model outputs an oil leakage defect result corresponding to the first real-time visible light image, and a heating defect result corresponding to the real-time fused image, wherein the real-time fused image is obtained by registering and fusing a second real-time visible light image of the transformer substation heating defect and a real-time infrared image; The heating defect result is mapped to the second real-time visible light image according to a mapping method to obtain the location of the infrared heating defect of the substation equipment.

2. A dual-mode defect identification method for substation equipment according to claim 1, characterized in that: The background fusion expansion of the first visible light image based on the preset improved YOLOv8 instance segmentation model is performed to obtain a substation equipment structural defect image dataset including: Performing instance segmentation on the first visible light image according to the improved YOLOv8 instance segmentation model to obtain an oil leakage image, and establishing a rectangular coordinate system in the oil leakage image; Solve the convex hull of the oil leakage image and divide the convex hull into Graphics, calculation The coordinates of the center of gravity of the figure and area , and calculate the centroid coordinates of the convex hull , the expression is: ， ， A rectangular coordinate system is established in the transformer image without oil leakage defects. The Canny operator is used to detect the edge of the ground where the transformer is located to obtain the ground edge position information. The area surrounded by the edge is defined as the ground area D in the form of a set of coordinate points. A centroid point is selected every preset number of pixels in the transformer image. , each centroid point is screened according to the preset constraint conditions to obtain m centroid points, and the expression of the constraint conditions is: ， ， ， In the formula, , For The horizontal and vertical coordinates of the points on the convex hull of the oil leakage graph are the centroids, is the horizontal coordinate of the point on the transformer target detection box; For each transformer image without oil leakage defect, only the first best center of gravity point is retained. and the second best center of gravity For image expansion, the expression of the best center of gravity selection rule is: ， In the formula, is the distance between any two points among the m centroids, , are the horizontal and vertical coordinates of the first optimal center of gravity, , are the horizontal and vertical coordinates of the second best center of gravity respectively; First best center of gravity and the second best center of gravity A new oil leakage defect image is generated from the transformer image without oil leakage defect as the center of gravity of the oil leakage image, and the new oil leakage defect image is added to the structural defect image dataset of substation equipment.

3. A dual-mode defect identification method for substation equipment according to claim 1, characterized in that: The improved YOLOv8 instance segmentation model and the improved YOLOv8 target detection model both include a feature extraction network, a feature fusion network and a detection network.

4. A dual-mode defect identification method for substation equipment according to claim 3, characterized in that: in, The structure of the feature extraction network includes, in sequence: a convolution module, a first C2f module, a first LWA module, a second C2f module, a second LWA module, a third C2f module, a third LWA module, a fourth C2f module and an SPPF module; The structure of the feature fusion network includes, in sequence: a first upsampling module, a first splicing module, a fifth C2f module, a second upsampling module, a second splicing module, a sixth C2f module, a fourth LWA module, a third splicing module, a seventh C2f module, a fifth LWA module, a fourth splicing module and an eighth C2f module; The detection network includes a Detect module; Among them, the structures of the first LWA module, the second LWA module, the third LWA module, the fourth LWA module and the fifth LWA module are exactly the same, and the specific operation process is: the image features entering the first LWA module, the second LWA module, the third LWA module, the fourth LWA module or the fifth LWA module are first divided into two parts, one of which is used to extract local features through an average pooling layer, and then processed by a 1×1 point-by-point convolution module, and then processed by an activation layer, and then input into the multiplication module after processing with the other image features for array multiplication, and then output after summing through the summation module, wherein the processing process of the other image features is a grouped convolution operation with a convolution kernel of 3.

5. A dual-mode defect identification method for substation equipment according to claim 4, characterized in that: in, The output of the second C2f module in the feature extraction network and the output of the second upsampling module in the feature fusion network are connected through a second splicing module; The output of the third C2f module in the feature extraction network and the output of the first upsampling module in the feature fusion network are connected through a first splicing module; The output of the SPPF module in the feature extraction network and the output of the fifth LWA module in the feature fusion network are connected through a fourth splicing module; The output of the fifth C2f module in the feature fusion network is connected with the output of the fourth LWA module in the feature fusion network through a third splicing module.

6. A dual-mode defect identification system for substation equipment, characterized in that: include: An expansion module is configured to obtain a first visible light image of a transformer oil leakage defect, and perform background fusion expansion on the first visible light image based on a preset improved YOLOv8 instance segmentation model to obtain a substation equipment structural defect image dataset; a fusion module configured to obtain a second visible light image and an infrared image of a heating defect of a substation equipment, register and fuse the second visible light image and the infrared image, and obtain a fused image data set; A training module, configured to iteratively train a preset improved YOLOv8 target detection model according to the substation equipment structural defect image dataset and the fused image dataset to obtain a dual-modal defect recognition model; an output module, configured to obtain a first real-time visible light image of the transformer and a real-time fused image of the transformer substation, input the first real-time visible light image and the real-time fused image into the dual-modal defect recognition model, and the dual-modal defect recognition model outputs an oil leakage defect result corresponding to the first real-time visible light image and a heating defect result corresponding to the real-time fused image, wherein the real-time fused image is obtained by registering and fusion of a second real-time visible light image of the transformer substation heating defect and a real-time infrared image; A mapping module is configured to map the heating defect result to the second real-time visible light image according to a mapping method to obtain the location of the infrared heating defect of the substation equipment.

7. An electronic device, characterized in that: include: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images