A high-voltage equipment defect positioning system and method based on multimodal image fusion

Abstract

The present invention relates to the technical field of power equipment defect monitoring, and specifically to a high-voltage equipment defect location system and method based on multimodal image fusion, comprising an ultraviolet trigger module, a modal synchronization module, a spatial positioning module, a structural recognition module, and a fusion diagnosis module. In the present invention, by performing quantitative threshold determination on the corona discharge signal in the ultraviolet image, it is possible to automatically calibrate the suspected defect area, ensure the consistency of the three-modal data in the spatial and temporal dimensions, improve the image feature alignment accuracy by comparing the three types of coordinates in the image: the conductor hot spot, the boundary geometry, and the spot center, and perform registration offset correction, and achieve automatic identification of the contamination level by extracting the spot distribution path of the insulator string and calculating the continuous distribution length, corresponding to the contamination level standard, constructing a multimodal image fusion feature block and assigning a risk ranking label, effectively improving the accuracy of defect location and the comprehensive assessment capability of the defect nature and impact degree.

Description

Technical Field

[0001] The present invention relates to the technical field of power equipment defect monitoring, and in particular to a high-voltage equipment defect locating system and method based on multimodal image fusion. Background Art

[0002] The technical field of power equipment defect monitoring involves the perception, analysis, and diagnosis of the operating status of various types of equipment in the power system, including fault identification, local overheating detection, partial discharge monitoring, and insulation aging assessment during the operation of high-voltage transmission equipment. By comprehensively utilizing sensing and acquisition methods, image recognition methods, and electrical measurement technologies, equipment operating data is acquired and processed in real time to effectively identify defects and assess their status, providing technical support for equipment maintenance and operational safety. The high-voltage equipment defect location system refers to a technology used to identify defective areas within high-voltage equipment, primarily targeting local abnormal conditions caused by electrical stress, mechanical stress, or environmental factors during the operation of high-voltage transmission equipment. Infrared thermal imaging is typically used to obtain a temperature distribution map of the equipment surface, or ultraviolet imaging is used to obtain images of partial discharge signals. Defect location is then determined and located through manual visual analysis or single image feature extraction, based on the spatial distribution characteristics of the temperature anomaly signal or optical signal.

[0003] Existing technologies mostly use single infrared or ultraviolet images to determine equipment defects, and only rely on temperature distribution maps or discharge image features for analysis. They are easily affected by environmental background interference and image deviations, and it is difficult to accurately identify the source of defects in complex operating scenarios. Especially in areas where multiple wires cross, parts with complex insulation structures, or conditions where discharge signals are weakened, single-modal images have problems such as unclear target boundaries or misjudged position offsets. In addition, manual interpretation methods rely on experience and judgment, which is prone to errors in scenarios with large data volumes or subtle feature differences, resulting in a lack of consistency and traceability in defect assessment results, which in turn affects the timeliness and accuracy of maintenance scheduling. Summary of the Invention

[0004] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a high-voltage equipment defect location system and method based on multimodal image fusion.

[0005] In order to achieve the above objectives, the present invention adopts the following technical solutions: A high-voltage equipment defect location system based on multimodal image fusion includes:

[0006] The UV trigger module acquires a sequence of UV channel images of high-voltage equipment on the transmission line and calculates the photon flux density in each frame. If the photon flux density in the current image exceeds the corona discharge intensity threshold, it records the timestamp and coordinate point and generates a broken strand corona trigger response coordinate set.

[0007] The modal synchronization module extracts infrared image and visible light image frames according to the broken strand corona trigger response coordinate set, selects image groups whose acquisition time differences of the three modal images are all less than the multi-source time accuracy, and generates a synchronized three-modal image group number set;

[0008] The spatial positioning module extracts the coordinates of the centroid of the ultraviolet image corona spot, the center point of the infrared image thermal field contour, and the centroid of the visible light image based on the number set of the synchronous trimodal image group, and determines whether the deviation of the three sets of coordinates in the pixel space exceeds the sub-pixel registration tolerance. If so, offset acquisition is performed to generate the alignment coordinates of the wire area image;

[0009] The structure recognition module obtains the adjusted trimodal image according to the alignment coordinates of the conductor area image, classifies and determines the defect type of the broken strand position of the transmission line conductor, determines the contamination level and records the number, and generates a broken strand type determination label and a contamination level identification number.

[0010] As a further solution of the present invention, the broken strand corona trigger response coordinate set includes the suspected broken strand position, image frame timestamp, and corona intensity mark coordinate point; the synchronous tri-modal image group number set includes the image combination number, tri-modal image synchronization timestamp, and valid modal identifier; the conductor area image alignment coordinates include the ultraviolet spot center coordinates, the thermal field contour center point coordinates, and the geometric boundary centroid coordinates; the broken strand type determination label includes the broken strand type mark, classification number, and conductor positioning index; the pollution level identification number is specifically the pollution level code, insulator boundary path index record, and continuous spot count value.

[0011] As a further solution of the present invention, the ultraviolet trigger module includes:

[0012] The image acquisition submodule obtains the UV channel image sequence of the high-voltage equipment on the transmission line, collects continuous image frames of the current conductor area, performs real-time synchronization and time stamp recording of the image frames, and generates a UV channel frame sequence set;

[0013] The image parameter extraction submodule extracts the image parameters of the wire area in each frame image based on the ultraviolet channel frame sequence set, determines the relationship between the pixel characteristics in the wire area and the corona discharge intensity threshold, identifies the location where the regional characteristics are prominent, and obtains the high-density wire pixel area;

[0014] The abnormal marking linkage submodule determines whether there is an abnormal response in the wire area based on the high-density wire pixel area. If so, the wire position in the corresponding frame image is marked as a suspected wire breakage area, and the timestamp corresponding to the image and the coordinate point of the marked area are extracted. A linkage record of the suspected wire breakage area and the image time is established to obtain the wire breakage corona trigger response coordinate set.

[0015] As a further solution of the present invention, the modal synchronization module includes:

[0016] The image extraction submodule obtains the coordinates and timestamp information in the broken strand corona trigger response coordinate set, searches for the time corresponding to the timestamp item by item, extracts the infrared image frames and visible light image frames at the corresponding time, records the number and corresponding time information of each image frame, combines them with the corona image frame, annotates the content of each combination, and generates a candidate trimodal image group dataset;

[0017] The time comparison submodule, based on the candidate trimodal image group dataset, performs synchronization information verification on a group-by-group basis according to the timestamp information of each image frame. By comparing the degree of synchronization deviation of the three types of image frames in time, it makes a judgment based on the multi-source time synchronization reference value, selects image combinations within the time deviation range, extracts the corresponding image number identifiers, and uniformly records them as valid data to obtain a valid image group number sequence;

[0018] The group number generation submodule performs unified naming standard processing on each number according to the valid image group number sequence, writes the image combination identifier into the modality synchronization image registration table, and updates the number information to the image management index structure to obtain a synchronous tri-modality image group number set.

[0019] As a further solution of the present invention, the spatial positioning module includes:

[0020] The image coordinate extraction submodule sequentially extracts the corona spot area in the ultraviolet image based on the image frames in the synchronous trimodal image group number set, obtains the coordinate position of the luminous centroid point in the image, locates the thermal field contour area corresponding to the conductor in the infrared image, obtains the thermal field center coordinates based on the temperature distribution boundary, and simultaneously identifies the intersection point formed by the geometric boundary lines of the conductor in the visible light image, extracts the location of the intersection point as the geometric center point of the conductor morphology, and records the key coordinate points of the three types of image frames in a unified format to generate a multi-source image centroid coordinate set;

[0021] The coordinate deviation judgment submodule compares the positional relationships of the three types of coordinate points (ultraviolet, infrared, and visible light) in the image space based on the centroid coordinate set of the multi-source image, determines whether there is a coordinate deviation between the three types, sets a sub-pixel registration tolerance, and marks a deviation group if the offset between any two types of image coordinates exceeds the tolerance value in any direction, and obtains the inter-frame offset judgment result;

[0022] The center adjustment submodule extracts the ultraviolet image coordinates as the reference acquisition center based on the deviation record items in the offset judgment results between the image frames, adjusts the center positioning of the infrared and visible light image acquisition systems in the direction of the ultraviolet coordinates according to the relative position relationship with the infrared image and the visible light image, completes the field of view calibration after the control instruction is executed, verifies the coordinates of the acquired image frames, records the adjusted three-image frame center alignment results, and establishes the wire area image alignment coordinates.

[0023] As a further solution of the present invention, the structure recognition module includes:

[0024] The image data acquisition submodule obtains the alignment coordinate information of the wire area image, sequentially retrieves the adjusted three-modal image data, sequentially locates the ultraviolet, infrared, and visible light image frames, performs regional image slicing operations based on the area where the coordinate points are located, extracts local image subframes of the corresponding area, annotates the image number, modality type, and coordinate position in the data structure, and groups the three types of images into a synchronized image group to generate a modality image subframe group set;

[0025] The strand break type identification submodule extracts the conductor structure area boundary based on the visible light image frames in the modal image subframe set and scans the pixel value changes along the boundary. It marks the fracture location and measures the discontinuous length of the conductor contour of the fractured segment. Combined with the thermal distribution changes in the fractured segment in the infrared image, it determines whether there is abnormal temperature diffusion. It calculates and obtains the fracture impact trend value, compares it with the classification level interval, and generates a strand break type determination label.

[0026] The pollution level judgment submodule extracts the image block where the insulator group in the substation area is located based on the ultraviolet image frame in the modal image subframe group set, determines the boundary path by identifying the outline of the insulator, scans the distribution of pixels where the discharge light spots appear on the boundary path in sequence, measures the number of consecutive light spots and records the proportion on the boundary curve, makes a judgment based on the pollution level standard classification threshold, establishes the corresponding pollution level record number, and obtains the pollution level identification number.

[0027] As a further embodiment of the present invention, the system further comprises:

[0028] The fusion diagnosis module uses the broken strand type determination label and contamination level identification number as the dominant area boundary, extracts the image feature sub-blocks at the corresponding position from the trimodal image, assigns a risk priority to each fused area, and sorts the contamination level numbers to form a cleaning priority list, generating multimodal fusion high-voltage equipment defect location data.

[0029] The multimodal fusion high-voltage equipment defect location data includes risk priority marks, image feature sub-block combinations, and cleaning priority lists.

[0030] As a further solution of the present invention, the fusion diagnosis module includes:

[0031] The image feature extraction submodule reads the position index recorded in the broken strand type determination label and the contamination level identification number as the dominant region boundary, reads the image coordinates corresponding to the boundary region from the trimodal image, and extracts the visible light image grayscale texture block, the infrared image temperature distribution block, and the ultraviolet image bright spot pixel block. After synchronously arranging the three types of image blocks according to the regional coordinates, a multimodal image joint region is formed to obtain the fused region image subblock;

[0032] The risk level assessment submodule identifies the feature values ​​of three types of image features, namely grayscale difference, temperature offset, and bright spot area distribution, extracted from the fused region image sub-blocks, and then performs standard rule matching to determine the abnormality level of each region in terms of grayscale, temperature, and bright spot ratio. Priority is assigned based on the aggregation relationship of the three types of abnormalities in spatial position and their relative distribution relationship with the central area of ​​the image. A correspondence between region numbers and priority numbers is established to obtain the regional risk priority.

[0033] The multimodal result generation submodule performs position sorting processing on all contamination level identification numbers according to the regional risk priority results, extracts the spatial index and modal image structure information corresponding to the numbers in each area, constructs an association mapping between the numbers and the image blocks and image frame numbers, and collects the image sub-block data and the corresponding position structure information in the order of the numbers to obtain multimodal fusion high-voltage equipment defect location data.

[0034] A method for locating defects in high-voltage equipment based on multimodal image fusion, comprising the following steps:

[0035] S1: Count the number of photons per unit area of ​​the conductor area in the UV image and compare it with the corona discharge intensity threshold. If the threshold is exceeded, the coordinates and time are recorded to generate a broken strand corona trigger response coordinate set;

[0036] S2: extracting infrared and visible light image frames according to the timestamp of the broken strand corona trigger response coordinates, comparing the time difference of the three modalities, screening image groups with errors less than the multi-source time accuracy, and generating a synchronized three-modal image group number set;

[0037] S3: extracting the coordinates of the three modal centers from the images in the synchronous three-modal image group number set, comparing the deviations of the ultraviolet coordinates with those of the infrared image and the visible light image, and controlling the offset acquisition if the deviations exceed the sub-pixel registration tolerance to generate the alignment coordinates of the wire area image;

[0038] S4: Based on the alignment coordinates of the conductor area image, the broken strand area and the length of the insulator spot chain are counted, the broken strand type is matched with the pollution level interval, and a broken strand type determination label and a pollution level identification number are generated;

[0039] S5: Based on the broken strand type determination label and the contamination level identification number, the trimodal block coordinates are matched, a risk priority is assigned to each fusion area, and the contamination level numbers are sorted to form a cleaning priority list to generate multimodal fusion high-voltage equipment defect location data.

[0040] Compared with the prior art, the advantages and positive effects of the present invention are:

[0041] In the present invention, by performing quantitative threshold judgment on the corona discharge signal in the ultraviolet image, automatic calibration of suspected defect areas can be achieved. By fusing the synchronously acquired infrared thermal image and visible light image and screening the effective image combination according to the image acquisition time accuracy, the consistency of the three-modal data in the spatial and temporal dimensions is ensured. By comparing the three types of coordinates of the conductor hot spot, boundary geometry and spot center in the image and performing alignment offset correction, the image feature alignment accuracy is improved. By extracting the spot distribution path of the insulator string and calculating the continuous distribution length, corresponding to the pollution level standard, automatic identification and numbering of the pollution level is achieved. By constructing a multimodal image fusion feature block and assigning a risk ranking label, a risk priority and cleaning order assessment result is formed, which effectively improves the accuracy of defect positioning and the comprehensive assessment capability of the defect nature and impact degree. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a system flow chart of the present invention;

[0043] Figure 2 This is a flow chart of the ultraviolet trigger module of the present invention;

[0044] Figure 3 This is a flow chart of the modal synchronization module of the present invention;

[0045] Figure 4 This is a flow chart of the spatial positioning module of the present invention;

[0046] Figure 5 This is a flow chart of the structure recognition module of the present invention;

[0047] Figure 6 This is a flow chart of the fusion diagnosis module of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0049] In the description of the present invention, it should be understood that the terms "length," "width," "up," "down," "front," "back," "left," "right," "vertical," "horizontal," "top," "bottom," "inside," "outside," and the like, indicating positions or relationships, are based on the positions or relationships shown in the accompanying drawings and are intended only to facilitate the description of the present invention and simplify the description. They do not indicate or imply that the devices or elements referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limiting the present invention. Furthermore, in the description of the present invention, "plurality" means two or more, unless otherwise expressly and specifically defined.

[0050] See also Figure 1 , a high-voltage equipment defect location system based on multimodal image fusion includes:

[0051] The UV trigger module acquires a sequence of UV channel images on high-voltage equipment on the transmission line, calculates the photon flux density in each frame, and compares it with the corona discharge intensity threshold. If the photon flux density of any conductor area in the current image exceeds the corona discharge intensity threshold (in compliance with the "Corona Discharge Detection Guidelines"), the corresponding location is marked as a suspected broken strand area, the image frame timestamp is extracted, and the coordinate points are recorded in a linked manner to generate a broken strand corona trigger response coordinate set.

[0052] The modal synchronization module extracts infrared and visible light image frames from the same moment based on the coordinates and timestamp information in the broken strand corona trigger response coordinate set. It then compares the acquisition time difference between the three modal images with the multi-source time accuracy (±1 μs, as set by the IEEE 1588-2008 PTP protocol). If the time difference between all images is less than the multi-source time accuracy, the image combination number is stored in the valid modal group list to generate a synchronized three-modal image group number set.

[0053] The spatial positioning module collects images based on the synchronized trimodal image group number, extracts the centroid coordinates of the corona spot in the ultraviolet image, the center point of the thermal field contour of the conductor temperature distribution map in the infrared image, and the centroid coordinates of the intersection of the conductor geometric boundary lines in the visible light image, and compares the deviation of the three sets of coordinates in the X and Y directions in the pixel space. If the deviation exceeds the sub-pixel registration tolerance (meeting the image resolution standard), the infrared and visible light image acquisition centers are controlled to move closer to the ultraviolet coordinates to perform offset acquisition, generating the conductor area image alignment coordinates.

[0054] The structure recognition module obtains an adjusted trimodal image based on the coordinates of the conductor area image, classifies the defect type at the location of the broken strand of the transmission line conductor, and simultaneously extracts the distribution path of the discharge light spot at the insulator boundary in the ultraviolet image of the insulator group in the substation. The module calculates the number of continuous light spots along the projected length of the insulator string surface, compares this with the contamination level classification, determines the contamination level, records the number, and generates a broken strand type determination label and contamination level identification number.

[0055] The fusion diagnosis module determines the label and contamination level identification number based on the broken strand type, uses the position corresponding to the broken strand label as the dominant area boundary, extracts the image feature sub-blocks at the corresponding position from the trimodal image, assigns a risk priority to each fused area (implementing the ISO 31000 risk management standard), and sorts the contamination level numbers to form a cleaning priority list, generating multimodal fusion high-voltage equipment defect location data.

[0056] The broken strand corona trigger response coordinate set includes the suspected broken strand position, image frame timestamp, and corona intensity mark coordinate point. The synchronous tri-modal image group number set includes the image combination number, tri-modal image synchronization timestamp, and effective mode identification. The conductor area image alignment coordinates include the ultraviolet spot center coordinates, the thermal field contour center point coordinates, and the geometric boundary centroid coordinates. The broken strand type determination label includes the broken strand type mark, classification number, and conductor positioning index. The contamination level identification number includes the contamination level code, insulator boundary path index record, and continuous spot count value. The multi-modal fusion high-voltage equipment defect location data includes the risk priority mark, image feature sub-block combination, and cleaning priority list.

[0057] See also Figure 2 , the UV trigger module includes:

[0058] The image acquisition submodule obtains the UV channel image sequence of the high-voltage equipment on the transmission line, collects continuous image frames of the current conductor area, performs real-time synchronization and time stamp recording of the image frames, and generates a UV channel frame sequence set;

[0059] When acquiring a sequence of UV channel images on high-voltage equipment on a transmission line, a UV imaging detection device installed on the tower crossarm or at the end of an insulator string is used. The acquisition frequency of this device is generally set to 30 frames per second. During continuous monitoring, images of the conductor area are acquired frame by frame, and their timestamp information is recorded. In actual operation, for a typical operating environment of a 220 kV line, the system continuously acquires four frames of UV images. Each frame captures an image matrix area centered on the conductor, and the system automatically identifies the pixel block where the conductor is located. For example, in the first frame, the number of pixels in the conductor area is 4500, the average brightness is 120.5 cd / m², the pixel change value is 25, and the image acquisition timestamp is 1000 ms. The basic parameter information of the image frame is listed below:

[0060] .

[0061] As shown in Table 1, each image frame records the core image attributes of the wire area. The brightness is expressed in cd / m², reflecting the illumination characteristics of the wire's excited area. The pixel change value is the pixel-level difference in the overlapping portion of the wire area with the previous frame. This difference can be obtained by statistically summing the grayscale differences of corresponding pixels between the image frame matrices. For example, there are approximately 4400 matching pixels in the wire area between the second and first frames, of which 1320 pixel values ​​change by more than the set minimum change step of 2 grayscale levels and are therefore classified as valid pixel changes. The final pixel change value is 40. After calibration, this image is used in the next stage of parameter extraction, ultimately generating a set of UV channel frame sequences.

[0062] The image parameter extraction submodule extracts the image parameters of the wire area in each frame image based on the UV channel frame sequence set, determines the relationship between the pixel characteristics in the wire area and the corona discharge intensity threshold, identifies the location where the regional characteristics are prominent, and obtains the high-density wire pixel area;

[0063] Based on the acquired UV channel frame sequence set, it is necessary to further extract image parameters from each frame image to support the identification of suspected discharge areas. This mainly involves steps such as brightness distribution extraction, conductor area stability analysis, and mutation boundary detection. First, the pixel brightness of each frame image in Table 1 is counted. The average brightness value is obtained by converting the grayscale values ​​of all conductor area pixels into photometric units and then taking the average. For example, in the third frame image, the average grayscale value of the 4550 conductor area pixels is 135, and the corresponding brightness conversion value is 118.7cd / m². Then the pixel change value is corrected. In the experiment, the change value of the third frame image is 30, which is within the set change critical zone of 25 to 35, indicating that the regional boundary fluctuation is slight, but does not exceed the significant fluctuation judgment standard. The brightness threshold range set by the system is 100 to 140cd / m². If the average brightness in the image frame is lower than 90 or higher than 150, it will be defined as a strong fluctuation area. The brightness of the second frame image is 130.2cd / m² and the change value is 40. The pixel change has exceeded the set upper limit of 35, and it is judged as an abnormal fluctuation frame. In the parameter extraction stage, the system compares the brightness and pixel change comparison rules as follows:

[0064] Brightness stability range: 100 ≤ brightness ≤ 140cd / m²;

[0065] Pixel change threshold: ≤35;

[0066] Abnormal identification criteria: If either the brightness or pixel change value exceeds the above range, the process proceeds to the suspected area extraction step. Taking frame 2 as an example, its brightness of 130.2cd / m² is within the normal range, but the pixel change value is 40, exceeding the threshold. Therefore, it is selected to enter the next stage of judgment range. After completing the parameter judgment, it is automatically marked as a high-density wire pixel area.

[0067] The abnormal marking linkage submodule determines whether there is an abnormal response in the conductor area based on the conductor pixel area. If so, the conductor position in the corresponding frame image is marked as a suspected strand break area, and the corresponding timestamp and coordinate points of the marked area are extracted. A linkage record of the suspected strand break area and the image time is established to obtain the strand break corona trigger response coordinate set;

[0068] Based on the extracted high-density conductor pixel areas, the system enters the abnormal response determination phase. This phase is based on the timestamp, image number, and conductor coordinate mapping table. The system first determines whether any regional characteristic value exceeds the threshold. If so, the image frame number is marked and the corresponding timestamp is searched. For example, if the time of frame 2 is 1030ms, the corresponding conductor number is determined to be conductor segment B3 after image analysis. The center of its pixel block is located at row 360, column 820 in the image matrix. Based on the camera device settings, this coordinate can be projected to the geographic coordinates X = 145.27m, Y = 47.93m. The system automatically forms a linkage triple structure with image number 2, timestamp 1030ms, and coordinate point X = 145.27m, Y = 47.93m and enters it into the response linked list. At the same time, if multiple image frames have the same abnormal characteristics, multiple record nodes are generated in parallel and uniformly archived in the abnormal trigger data set. Finally, a record set of conductor abnormal response points is constructed and output as a broken strand corona trigger response coordinate set.

[0069] See also Figure 3 , the modal synchronization module includes:

[0070] The image extraction submodule obtains the coordinates and timestamp information in the broken strand corona trigger response coordinate set, searches for the time corresponding to the timestamp item by item, extracts the infrared image frames and visible light image frames at the corresponding time, records the number and corresponding time information of each image frame, combines them with the corona image frame, annotates the content of each combination, and generates a candidate trimodal image group dataset;

[0071] To obtain the coordinate and timestamp information in the broken strand corona trigger response coordinate set, it is necessary to traverse the records in the coordinate set one by one, locate the time point of the corona trigger image frame in the image acquisition system by the timestamp, and query the infrared image and visible light image frame resources based on the time point. The sampling frequency set by the image acquisition system is 100 frames per second, the image frame time resolution is 10μs, and the system sets the time index window to ±10μs. For each corona image timestamp, search for image frames with a time difference less than the window range in the infrared image cache and the visible light image cache. If the corona The image timestamp is 1000000μs. For infrared and visible light images, the image frames within the range of 999990μs to 1000010μs must be retrieved respectively. Assuming that the timestamp of the retrieved infrared image IR_001 is 1000001μs and the timestamp of the retrieved visible light image VIS_001 is 999999μs, the three frames constitute a modal combination. The system generates a unique number M001 for them and records the timestamps and number indexes of the three frames. All modal combinations that meet the matching conditions are then generated into a registration table of number information and timestamps, which are summarized as follows:

[0072] .

[0073] As shown in Table 2, modality groups M001 to M004 all complete the three-frame image timestamp recording and combination numbering operations to generate candidate three-modality image group datasets.

[0074] The time comparison submodule is based on the candidate trimodal image group dataset and performs synchronization information verification on a group-by-group basis according to the timestamp information of each image frame. By comparing the degree of synchronization deviation of the three types of image frames in time, it makes a judgment based on the multi-source time synchronization benchmark value, selects image combinations within the time deviation range, extracts the corresponding image number identifiers, and uniformly records them as valid data to obtain a valid image group number sequence;

[0075] Based on the candidate trimodal image group dataset, the system reads the three timestamp fields of each group of image frames in turn, performs time synchronization offset judgment, and calculates the maximum time difference of the three frames of image for each group number as the synchronization offset judgment criterion. The system sets the synchronization judgment standard as the offset does not exceed 1μs, that is, when the maximum absolute value of the time difference of the three frames of image is less than or equal to 1μs, it is considered to be a synchronized group. Taking M001 as an example, its corona image is 1000000μs, the infrared image is 1000001μs, and the visible light image is 999999μs. The maximum time offset is 2μs, which exceeds the allowable range. Therefore, the group is judged invalid. The offsets of M002 to M004 are all within 1μs, which meets the synchronization standard. The system writes the qualified numbers into the valid number cache list and marks the judgment result and the corresponding offset. The specific judgment results are shown in the following table:

[0076] .

[0077] As shown in Table 3, M002 to M004 meet the 1μs synchronization tolerance standard. The screening results are written into the cache and enter the number allocation process to obtain a valid image group number sequence.

[0078] The group number generation submodule processes the numbers in a unified naming standard according to the valid image group number sequence, writes the image combination identifier into the modality synchronization image registration table, and updates the number information into the image management index structure to obtain the synchronized tri-modality image group number set;

[0079] According to the valid image group number sequence, the system will uniformly archive the three groups of numbers from M002 to M004. The numbering format will retain the M+three-digit naming. All image group numbers will establish a correspondence with the actual image storage path. For example, the image frame path corresponding to the M002 number is / IR / IR_002.jpg, / VIS / VIS_002.jpg, and / UV / UV_002.jpg. The mapping relationship is completed by establishing a number mapping table, and an index table entry is established in the image management database. The index table content must include four fields: modality group number, three-frame image path, acquisition timestamp, and whether synchronization is valid. Subsequent data call operations can directly retrieve the image path information through the index number, and at the same time output the information as a standard data structure format XML file and archive it. After the entire process is completed, the system records and encodes the numbering results summary, which is the synchronized three-modality image group number set.

[0080] See also Figure 4 , the spatial positioning module includes:

[0081] The image coordinate extraction submodule extracts the corona spot area in the ultraviolet image based on the image frames in the synchronous trimodal image group number set, obtains the coordinate position of the luminous centroid point in the image, locates the thermal field contour area corresponding to the conductor in the infrared image, obtains the coordinates of the thermal field center based on the temperature distribution boundary, and simultaneously identifies the intersection point formed by the geometric boundary lines of the conductor in the visible light image, extracts the location of the intersection point as the geometric center point of the conductor morphology, and records the key coordinate points of the three types of image frames in a unified format to generate a multi-source image centroid coordinate set;

[0082] Based on the image frames in the synchronous trimodal image group number set, the center coordinates of the key feature areas are extracted from the ultraviolet, infrared and visible light images respectively. First, the corona spot area is selected in the ultraviolet image. The system binarizes the image according to the grayscale intensity threshold, extracts the row and column indexes of all pixels in the spot brightness concentration area, and calculates its weighted center value in the row (Y direction) and column (X direction) as the centroid coordinates of the ultraviolet image. For example, if the center of the spot is concentrated on the row and column points between X=525 and X=535, and the grayscale values ​​are symmetrically distributed, the center coordinates calculated by the system are approximately X=530, Y=412. Then, the temperature threshold is segmented for the wire area in the infrared image, and the pixels with a temperature greater than or equal to 50 are extracted. ℃ closed thermal field contour, forms a polygonal envelope structure by identifying the boundary point set, and obtains its geometric center point to further identify the wire structure boundary in the visible light image. The system extracts edge polyline nodes based on the grayscale gradient edge method. For example, the intersection of the left and right boundaries of the wire is detected at positions X=600 and X=620, and the midpoint X=610 is calculated as the centroid of the wire geometric boundary. The Y coordinate is obtained from the intersection of the vertical lines. To ensure the comparability of the centroid coordinates, the system uniformly converts the three types of images to the same image resolution standard with a unified resolution of 0.1mm / pixel. After the conversion, the centroid coordinates are reassigned to pixel units, and a numbered index structure table is established to collect the coordinate point contents under all image groups to generate a multi-source image centroid coordinate set.

[0083] The coordinate deviation judgment submodule compares the positional relationships of the three types of coordinate points (ultraviolet, infrared, and visible light) in the center of mass coordinate set of the multi-source image in the X and Y directions in the image space, determines whether there is a coordinate deviation between the three, and sets the sub-pixel registration tolerance. If the offset between any two types of image coordinates exceeds the tolerance value in any direction, it is marked as a deviation group, and the inter-frame offset judgment result is obtained;

[0084] According to each set of data in the multi-source image centroid coordinate set, the system compares the centroid coordinates of the three image frames of ultraviolet, infrared and visible light to determine their relative position offset in the pixel space. During execution, the X and Y directions are used as two independent dimensions for processing, and the centroid coordinate values ​​of the three types of images are analyzed for pairwise difference. If the deviation of any pair of coordinates in a single direction exceeds the set tolerance threshold, it is marked as not meeting the registration requirements. The system sets the tolerance threshold to 0.5 pixels based on the requirements for sub-pixel accuracy in the national standard image recognition resolution. This tolerance value is widely used in most industrial detection scenarios. The setting of the registration tolerance is based on the pixel position control experiment of the high-resolution image acquisition system on the photographed object under a stable support environment. In the experiment, 10 For a group of corona images, the actual center of mass position is obtained by manual annotation and paired with the corresponding points in the infrared and visible light images to obtain the deviation value distribution range. 80% of the sample deviations are less than or equal to 0.5 pixels, so this is used as the judgment benchmark. For example, in a certain image group, the ultraviolet coordinate is X=500.2, the infrared is X=500.8, and the visible light is X=501.0. The deviation between the infrared and ultraviolet is 0.6 pixels, which exceeds the threshold and is marked as requiring adjustment. The system records the deviation value and image group number and establishes a deviation list. The list also includes three structure fields: deviation direction, deviation value size, and deviation pair type (UV-IR, IR-VIS, VIS-UV). Finally, the judgment basis for whether there is an offset between image groups is established, and the offset judgment result between image frames is obtained.

[0085] The center adjustment submodule extracts the ultraviolet image coordinates as the reference acquisition center based on the deviation record items in the offset judgment results between image frames. Based on the relative position relationship with the infrared image and visible light image, the center positioning of the infrared and visible light image acquisition systems is adjusted in the direction of the ultraviolet coordinates. After the control instructions are executed, the field of view calibration is completed, the coordinates of the acquired image frames are verified, and the adjusted three-image frame center alignment results are recorded to establish the image alignment coordinates of the wire area.

[0086] According to the image group number marked as needing adjustment in the image frame offset judgment result, the system sequentially retrieves the centroid coordinate information recorded in the corresponding ultraviolet image of the group and uses it as the current reference acquisition center. The infrared and visible light acquisition modules make slight adjustments in the X and Y directions based on the relative difference between the current centroid and the center of their own acquisition field of view. The motor mechanism is controlled to drive the lens module to perform a translation operation to ensure the accuracy of the offset operation. The system sets the minimum adjustment step unit to 0.1 pixel, corresponding to a physical offset of 0.01mm. During execution, the offset data is decomposed into integer and decimal parts, and the large-step adjustment and micro-step compensation methods are used. The movement is performed segment by segment. For example, if the deviation of the infrared image in the X direction is 0.6 pixels, the first step is to perform a reference movement of 0.5 pixels, and the second step is to perform a compensation offset of 0.1 pixels. The same is true for the Y direction. After the execution is completed, a new image frame is collected again and the coordinates are recalculated. The infrared and visible light coordinates of the new image are matched with the ultraviolet centroid coordinates. If the deviation values ​​of the center point coordinates of the three types of image frames in both the X and Y directions are less than 0.5 pixels, the system records it as a successful alignment, and establishes a new coordinate structure for the center points of the three types of images under the image group number. The structure is written into the wire area image registration coordinate table to generate the wire area image alignment coordinates.

[0087] See also Figure 5 , the structure recognition module includes:

[0088] The image data acquisition submodule obtains the alignment coordinate information of the conductor area image, sequentially retrieves the adjusted three-modal image data, locates the ultraviolet, infrared, and visible light image frames in sequence, performs regional image slicing operations based on the area where the coordinate points are located, extracts the local image subframes of the corresponding area, annotates the image number, modality type, and coordinate position in the data structure, and combines the three types of images into a synchronized image group to generate a modal image subframe group set;

[0089] After obtaining the coordinate information of the wire area image alignment, the system locates the three types of image frames: ultraviolet image, infrared image, and visible light image according to the image number and coordinate position field content contained in the coordinates. Then, the regional image is cropped with the coordinate point of each group of image frames as the center, and the cropping radius is set to 30 pixels. That is, the image subframe block is constructed by extending 30 pixels up, down, left, and right at the coordinate point to form an image area slice. At the same time, the image frame number, modality type, coordinate point index, and subframe boundary index are uniformly recorded as structured fields. The system further judges the legitimacy of the cropped area boundary and excludes image groups that are out of bounds due to coordinates close to the edge. Then, the image subframes are matched and combined according to the number to generate a synchronized three-modal image subframe group. Each subframe group structure contains three frames of images. The system records the image width and height, modality attributes, frame sequence number, and region boundary index, and uniformly incorporates them into the image frame data cache. After processing, the following sample data set is formed:

[0090] .

[0091] As shown in Table 4, this structure ensures spatial alignment of the three-modal images through a unified region parameter extraction method. The resolution of all subframes is set to the pixel ratio of the original image. By default, the image frame size is 1024×768 pixels, and the region size is fixed to 60×60 pixels. The above structure is used to delineate the three-modal alignment region and ultimately generate a set of modal image subframes.

[0092] The broken strand type identification submodule extracts the conductor structure area boundary based on the visible light image frames in the modal image subframe group and scans the pixel value changes along the boundary. It marks the fracture location and measures the discontinuous length of the conductor contour of the fractured segment. Combined with the thermal distribution changes in the fractured segment in the infrared image, it determines whether there is abnormal temperature diffusion. The formula is used:

[0093] ;

[0094] The fracture impact trend value is obtained by calculation and compared with the classification level interval to generate a fracture type determination label; Indicates the fracture impact trend value, Indicates the The pixel length of the broken segment, in pixels, represents the average pixel length of all fractured segments, Indicates the The highest pixel temperature value of each fault segment in the infrared image, in degrees Celsius (℃), Indicates the average temperature of the background area of ​​the conductor where the fracture section is located, in degrees Celsius (℃). Indicates the The shortest pixel distance from the centroid of a fracture segment to the edge of the image, in pixels. Indicates the maximum possible edge distance in the image frame, in pixels. Indicates the number of all broken segments in the image;

[0095] Based on each set of trimodal image frame data in the modal image subframe set, the system first extracts the wire boundary structure in the visible light image. It then detects fracture areas by scanning the difference in pixel values ​​of the contours within the region. The coordinates of the start and end points of each fracture are recorded, and the pixel length of each fracture segment is calculated based on its span in the X or Y direction. The system then extracts the maximum thermal value of the corresponding region in the infrared image for each fracture segment as the peak thermal value of that segment. The average temperature of the wire region in the adjacent region of the fracture segment is used as the background thermal value to construct a thermal value difference term. The shortest pixel distance from the centroid of the fracture segment to the edge of the image is also recorded. The following is the actual data record of three fracture structures in a certain image set:

[0096] .

[0097] According to the data in Table 5, the system substitutes the above values ​​into the trend formula, and the specific calculation process is as follows:

[0098] Paragraph 1: , , the product is 0.00735;

[0099] Paragraph 2: , , the product is 0.01379;

[0100] Paragraph 3: , , the product is 0;

[0101] The sum of the numerators is: 0.00735+0.01379=0.02114;

[0102] The denominator is calculated as:

[0103] ;

[0104] The overall trend value is:

[0105] ;

[0106] The system is set according to the judgment interval: the mild interval is , the moderate range is , severe , the current result falls into the mild range, the system generates a corresponding number and creates a label, and finally obtains the broken stock type determination label.

[0107] The fracture impact trend value is a dimensionless indicator used to measure the comprehensive anomaly degree of multiple fracture segments in a transmission line in three dimensions: morphological structure, thermal anomaly, and spatial distribution. This value comprehensively considers the degree of deviation of each fracture segment relative to the overall average fracture length, the increase in its local infrared temperature relative to the background temperature, and the relative position of the fracture segment to the central area in the image space. It quantifies the anomaly intensity of each fracture segment through the combined effect of structural differences and thermal value fluctuations, and adjusts the overall anomaly trend through spatial position weighting. A larger value indicates a more extreme fracture segment in terms of length deviation, temperature increase, and concentration, reflecting a stronger impact of the fracture structure on line stability. Therefore, it serves as an important basis for determining the severity of the fracture.

[0108] The overall operation logic of the formula is based on the comprehensive quantitative relationship between the length deviation degree of the wire fracture section, the degree of thermal value anomaly and the spatial distribution characteristics. The product between the structural normalization deviation and the thermal increase is used to reflect the importance of a single fracture section. Obtain the relative fluctuation of the fracture length, capture the difference between the fracture morphology and the average state, and then compare it with the relative increase in calorific value. Multiplying them together reflects the weight of the fracture segments that have both structural significance and temperature anomalies. After summing all the fracture segments, the cumulative performance of the structural thermal coupling strength is obtained as the numerator; the denominator introduces As an adjustment item for the spatial distribution of the fracture segments, this part is used to measure whether the fracture segments deviate from the center of the image. If they are mostly concentrated at the edge of the image, this item is increased to reduce the overall trend value, thereby suppressing the edge noise. At the same time, the bottom growth is balanced by the constant 1 to avoid the denominator being too small. The square root term is not introduced in the entire formula to maintain dimensional consistency and simplify the structure. In this structure, multiplication is used to construct joint importance, summation is used for multi-segment aggregation, and fractional form is used for standardization. The final output trend value It is a dimensionless comprehensive index that can reflect the overall abnormality of the fault segment in terms of structural changes, temperature performance and image space.

[0109] The pollution level judgment submodule extracts the image block of the substation area insulator group based on the ultraviolet image frame in the modal image subframe group set, determines the boundary path by identifying the outline of the insulator, and sequentially scans the distribution of pixels where the discharge light spots appear on the boundary path. It measures the number of consecutive light spots and records their proportion on the boundary curve. It then compares this with the pollution level standard classification threshold, establishes the corresponding pollution level record number, and obtains the pollution level identification number.

[0110] The UV images in the modal image subframe group are processed to detect the edge contour path of the insulator string in the image and establish a boundary line. The pixel values ​​of the highlight area are identified from this path. Pixels with a grayscale threshold of 230 or above in the image are set as discharge spot pixels. Continuous pixel groups are extracted along the insulator path, and the path length covered by the continuous bright spot is calculated as the discharge length. The total length of the insulator path is then recorded and the proportion factor is constructed. The following is the actual observation data of a certain image group:

[0111] .

[0112] According to the values ​​in Table 6, the system adopts the pollution level classification standard:

[0113] A discharge ratio >0.60 was considered severe;

[0114] A discharge ratio of 0.30–0.60 was judged as moderate;

[0115] A discharge ratio <0.30 was considered mild.

[0116] As shown in Table 6, the light spot ratio is 0.70, which is judged as a heavy pollution level. The system mark number is S3. The corresponding image number and judgment value are recorded, and the pollution level identification number is finally obtained.

[0117] See also Figure 6 , the fusion diagnosis module includes:

[0118] The image feature extraction submodule uses the broken strand type determination label and contamination level identification number to read the position index recorded in the broken strand label as the dominant region boundary. It then reads the image coordinates corresponding to the boundary region from the trimodal image and extracts the visible light image grayscale texture blocks, infrared image temperature distribution blocks, and ultraviolet image bright spot pixel blocks. These three types of image blocks are synchronously arranged according to the regional coordinates to form a multimodal image joint region, and the fused region image subblock is obtained.

[0119] Based on the broken strand type determination label and the contamination level identification number, the system sequentially extracts the coordinate region boundary information recorded in the broken strand label and the image sequence corresponding to the number. It extracts the grayscale blocks intersecting the break position from the visible light image in the order of the image frame number and records the maximum, minimum, and mean values ​​of the grayscale pixel matrix. It extracts the maximum temperature value in the corresponding area of ​​the infrared image and collects the average value of the background temperature of the area. It extracts the distribution boundary of the bright spot pixels from the ultraviolet image and counts the pixel area size. After the image feature structure is unified and merged through the above three-modal image processing process, the system constructs a fused image region table with the region number as the index, as shown in Table 7.

[0120] .

[0121] As shown in Table 7, the average grayscale value in region Z001 is 142, the peak temperature is 76.3°C, the background temperature is 68.0°C, and the UV bright spot area is 52 pixels². The grayscale value in region Z002 is 128, the peak temperature is 69.8°C, and the bright spot area is 38 pixels². The grayscale value in region Z003 is 137, the temperature is 73.2°C, and the bright spot area is 44 pixels². The above data are obtained through the trimodal image processing process, and finally the fused regional image sub-block is obtained.

[0122] The risk level assessment submodule identifies the characteristic values ​​of three types of image features, namely grayscale difference, temperature offset, and bright spot area distribution, extracted from the fused regional image sub-blocks. It then performs standard rule matching to determine the abnormality level of each region in terms of grayscale, temperature, and bright spot ratio. It then prioritizes the three types of abnormalities based on their spatial aggregation and relative distribution to the central area of ​​the image. It then establishes a correspondence between region numbers and priority numbers to determine the regional risk priority.

[0123] Based on the characteristic parameter information recorded in the sub-blocks of the fusion area image, the system first compares the grayscale value of each sub-block with the global grayscale reference value of the image, sets the grayscale reference value to 125, and the grayscale judgment threshold to 15. The grayscale of area Z001 is 142, and the deviation is 17, which is greater than the threshold and is judged as a grayscale offset area. The grayscale of Z002 is 128, and the deviation is 3, which does not reach the threshold. The grayscale of Z003 is 137, and the deviation is 12, which does not reach the threshold. Secondly, the difference between the temperature peak and the background temperature is judged. The temperature difference judgment threshold is 5℃. The temperature difference of Z001 is 8.3℃, the temperature difference of Z002 is 1.8℃, and the temperature difference of Z003 is 5.2℃. Z001 and Z003 meet the judgment conditions. Z001 has 52 pixels², accounting for 52% and meets the conditions. Z002 has 38 pixels², accounting for 38% and does not meet the conditions. Z003 has 44 pixels², accounting for 44% and meets the conditions. The number of conditions that each region meets in the three dimensions is used as the basis for judging the priority level. The value of 1 is assigned to those that meet 3 items, 2 to those that meet 2 items, and 3 to those that meet 1 item. Region Z001 meets 3 items and is numbered 1, Z003 meets 2 items and is numbered 2, and Z002 meets 1 item and is numbered 3. Finally, the regional risk priority is obtained.

[0124] The multimodal result generation submodule positions all contamination level identification numbers according to the regional risk priority results, extracts the spatial index and modal image structure information corresponding to the numbers in each area, constructs an association mapping between the numbers and the image block and image frame numbers, and collects the image sub-block data and corresponding position structure information in the order of the numbers to obtain multimodal fusion high-voltage equipment defect location data;

[0125] Based on the regional risk priority, the system reads the pollution level identification number, modal image type, and spatial position index corresponding to each number in numerical order, extracts the image frame number, regional boundary coordinates, identification number, and modal structure, and unifies the identification number with the corresponding modal image and spatial position to construct a data record structure. For example, number Z001 corresponds to image frame number VIS_043, with center coordinates of X=460, Y=520, pollution level number S3, image modality of visible light, and risk level 1. Z002 corresponds to VIS_045, X=420, Y=585, and pollution level S2. Z003 corresponds to VIS_046, X=448, Y=600, and pollution level S3. The system constructs the above structure into a positioning data structure and outputs it, ultimately obtaining multi-modal fusion high-voltage equipment defect location data.

[0126] A method for locating defects in high-voltage equipment based on multimodal image fusion, comprising the following steps:

[0127] S1: Count the number of photons per unit area of ​​the conductor area in the UV image and compare it with the corona discharge intensity threshold. If the threshold is exceeded, the coordinates and time are recorded to generate a broken strand corona trigger response coordinate set;

[0128] S2: Extract infrared and visible light image frames based on the timestamp of the broken strand corona trigger response coordinates, compare the time difference of the three modalities, select the image group with an error less than the multi-source time accuracy, and generate a synchronized three-modal image group number set;

[0129] S3: Extract the coordinates of the three modal centers from the images in the synchronized three-modal image group number set, compare the deviations of the ultraviolet coordinates with those of the infrared image and the visible light image, and control the offset acquisition if the deviations exceed the sub-pixel registration tolerance to generate the alignment coordinates of the wire area image;

[0130] S4: Based on the alignment coordinates of the conductor area image, the broken strand area and the length of the insulator spot chain are counted, the broken strand type is matched with the pollution level range, and the broken strand type determination label and pollution level identification number are generated;

[0131] S5: Based on the broken strand type determination label and the contamination level identification number, match the coordinates of the three-modal blocks, assign risk priority to each fusion area, and sort the contamination level numbers to form a cleaning priority list to generate multi-modal fusion high-voltage equipment defect location data.

[0132] The above are merely preferred embodiments of the present invention and do not limit the present invention in any other form. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A high-voltage equipment defect location system based on multimodal image fusion, characterized in that: The system comprises: The UV trigger module acquires a sequence of UV channel images of high-voltage equipment on the transmission line and calculates the photon flux density in each frame. If the photon flux density in the current image exceeds the corona discharge intensity threshold, it records the timestamp and coordinate point and generates a broken strand corona trigger response coordinate set. The modal synchronization module extracts infrared image and visible light image frames according to the broken strand corona trigger response coordinate set, selects image groups whose acquisition time differences of the three modal images are all less than the multi-source time accuracy, and generates a synchronized three-modal image group number set; The spatial positioning module extracts the coordinates of the centroid of the ultraviolet image corona spot, the center point of the infrared image thermal field contour, and the centroid of the visible light image based on the number set of the synchronous trimodal image group, and determines whether the deviation of the three sets of coordinates in the pixel space exceeds the sub-pixel registration tolerance. If so, offset acquisition is performed to generate the alignment coordinates of the wire area image; The structure recognition module obtains the adjusted trimodal image according to the alignment coordinates of the conductor area image, classifies and determines the defect type of the broken strand position of the transmission line conductor, determines the contamination level and records the number, and generates a broken strand type determination label and a contamination level identification number.

2. The high-voltage equipment defect location system based on multimodal image fusion according to claim 1 is characterized in that: The broken strand corona trigger response coordinate set includes the suspected broken strand position, image frame timestamp, and corona intensity mark coordinate point; the synchronous tri-modal image group number set includes the image combination number, tri-modal image synchronization timestamp, and valid modality identifier; the conductor area image alignment coordinates include the ultraviolet spot center coordinates, the thermal field contour center point coordinates, and the geometric boundary centroid coordinates; the broken strand type determination label includes the broken strand type mark, classification number, and conductor positioning index; the contamination level identification number is specifically the contamination level code, insulator boundary path index record, and continuous spot count value.

3. The high-voltage equipment defect location system based on multimodal image fusion according to claim 1 is characterized in that: The ultraviolet trigger module includes: The image acquisition submodule obtains the UV channel image sequence of the high-voltage equipment on the transmission line, collects continuous image frames of the current conductor area, performs real-time synchronization and time stamp recording of the image frames, and generates a UV channel frame sequence set; The image parameter extraction submodule extracts the image parameters of the wire area in each frame image based on the ultraviolet channel frame sequence set, determines the relationship between the pixel characteristics in the wire area and the corona discharge intensity threshold, identifies the location where the regional characteristics are prominent, and obtains the high-density wire pixel area; The abnormal marking linkage submodule determines whether there is an abnormal response in the wire area based on the high-density wire pixel area. If so, the wire position in the corresponding frame image is marked as a suspected wire breakage area, and the timestamp corresponding to the image and the coordinate point of the marked area are extracted. A linkage record of the suspected wire breakage area and the image time is established to obtain the wire breakage corona trigger response coordinate set.

4. The high-voltage equipment defect location system based on multimodal image fusion according to claim 1 is characterized in that: The modal synchronization module includes: The image extraction submodule obtains the coordinates and timestamp information in the broken strand corona trigger response coordinate set, searches for the time corresponding to the timestamp item by item, extracts the infrared image frames and visible light image frames at the corresponding time, records the number and corresponding time information of each image frame, combines them with the corona image frame, annotates the content of each combination, and generates a candidate trimodal image group dataset; The time comparison submodule, based on the candidate trimodal image group dataset, performs synchronization information verification on a group-by-group basis according to the timestamp information of each image frame. By comparing the degree of synchronization deviation of the three types of image frames in time, it makes a judgment based on the multi-source time synchronization reference value, selects image combinations within the time deviation range, extracts the corresponding image number identifiers, and uniformly records them as valid data to obtain a valid image group number sequence; The group number generation submodule performs unified naming standard processing on each number according to the valid image group number sequence, writes the image combination identifier into the modality synchronization image registration table, and updates the number information to the image management index structure to obtain a synchronous tri-modality image group number set.

5. The high-voltage equipment defect location system based on multimodal image fusion according to claim 1 is characterized in that: The spatial positioning module includes: The image coordinate extraction submodule sequentially extracts the corona spot area in the ultraviolet image based on the image frames in the synchronous trimodal image group number set, obtains the coordinate position of the luminous centroid point in the image, locates the thermal field contour area corresponding to the conductor in the infrared image, obtains the thermal field center coordinates based on the temperature distribution boundary, and simultaneously identifies the intersection point formed by the geometric boundary lines of the conductor in the visible light image, extracts the location of the intersection point as the geometric center point of the conductor morphology, and records the key coordinate points of the three types of image frames in a unified format to generate a multi-source image centroid coordinate set; The coordinate deviation judgment submodule compares the positional relationships of the three types of coordinate points (ultraviolet, infrared, and visible light) in the image space based on the centroid coordinate set of the multi-source image, determines whether there is a coordinate deviation between the three types, sets a sub-pixel registration tolerance, and marks a deviation group if the offset between any two types of image coordinates exceeds the tolerance value in any direction, and obtains the inter-frame offset judgment result; The center adjustment submodule extracts the ultraviolet image coordinates as the reference acquisition center based on the deviation record items in the offset judgment results between the image frames, adjusts the center positioning of the infrared and visible light image acquisition systems in the direction of the ultraviolet coordinates according to the relative position relationship with the infrared image and the visible light image, completes the field of view calibration after the control instruction is executed, verifies the coordinates of the acquired image frames, records the adjusted three-image frame center alignment results, and establishes the wire area image alignment coordinates.

6. The high-voltage equipment defect location system based on multimodal image fusion according to claim 1 is characterized in that: The structure recognition module includes: The image data acquisition submodule obtains the alignment coordinate information of the wire area image, sequentially retrieves the adjusted three-modal image data, sequentially locates the ultraviolet, infrared, and visible light image frames, performs regional image slicing operations based on the area where the coordinate points are located, extracts local image subframes of the corresponding area, annotates the image number, modality type, and coordinate position in the data structure, and groups the three types of images into a synchronized image group to generate a modality image subframe group set; The strand break type identification submodule extracts the conductor structure area boundary based on the visible light image frames in the modal image subframe set and scans the pixel value changes along the boundary. It marks the fracture location and measures the discontinuous length of the conductor contour of the fractured segment. Combined with the thermal distribution changes in the fractured segment in the infrared image, it determines whether there is abnormal temperature diffusion. It calculates and obtains the fracture impact trend value, compares it with the classification level interval, and generates a strand break type determination label. The pollution level judgment submodule extracts the image block where the insulator group in the substation area is located based on the ultraviolet image frame in the modal image subframe group set, determines the boundary path by identifying the outline of the insulator, scans the distribution of pixels where the discharge light spots appear on the boundary path in sequence, measures the number of consecutive light spots and records the proportion on the boundary curve, makes a judgment based on the pollution level standard classification threshold, establishes the corresponding pollution level record number, and obtains the pollution level identification number.

7. The high-voltage equipment defect location system based on multimodal image fusion according to claim 1 is characterized in that: The system further comprises: The fusion diagnosis module uses the broken strand type determination label and contamination level identification number as the dominant area boundary, extracts the image feature sub-blocks at the corresponding position from the trimodal image, assigns a risk priority to each fused area, and sorts the contamination level numbers to form a cleaning priority list, generating multimodal fusion high-voltage equipment defect location data. The multimodal fusion high-voltage equipment defect location data includes risk priority marks, image feature sub-block combinations, and cleaning priority lists.

8. The high-voltage equipment defect location system based on multimodal image fusion according to claim 7 is characterized in that: The fusion diagnosis module includes: The image feature extraction submodule reads the position index recorded in the broken strand type determination label and the contamination level identification number as the dominant region boundary, reads the image coordinates corresponding to the boundary region from the trimodal image, and extracts the visible light image grayscale texture block, the infrared image temperature distribution block, and the ultraviolet image bright spot pixel block. After synchronously arranging the three types of image blocks according to the regional coordinates, a multimodal image joint region is formed to obtain the fused region image subblock; The risk level assessment submodule identifies the feature values ​​of three types of image features, namely grayscale difference, temperature offset, and bright spot area distribution, extracted from the fused region image sub-blocks, and then performs standard rule matching to determine the abnormality level of each region in terms of grayscale, temperature, and bright spot ratio. Priority is assigned based on the aggregation relationship of the three types of abnormalities in spatial position and their relative distribution relationship with the central area of ​​the image. A correspondence between region numbers and priority numbers is established to obtain the regional risk priority. The multimodal result generation submodule performs position sorting processing on all contamination level identification numbers according to the regional risk priority results, extracts the spatial index and modal image structure information corresponding to the numbers in each area, constructs an association mapping between the numbers and the image blocks and image frame numbers, and collects the image sub-block data and the corresponding position structure information in the order of the numbers to obtain multimodal fusion high-voltage equipment defect location data.

9. A method for locating defects in high-voltage equipment based on multimodal image fusion, characterized in that: The method is used to implement a high-voltage equipment defect location system based on multimodal image fusion as described in any one of claims 1 to 8, comprising the following steps: S1: Count the number of photons per unit area of ​​the conductor area in the UV image and compare it with the corona discharge intensity threshold. If the threshold is exceeded, the coordinates and time are recorded to generate a broken strand corona trigger response coordinate set; S2: extracting infrared and visible light image frames according to the timestamp of the broken strand corona trigger response coordinates, comparing the time difference of the three modalities, screening image groups with errors less than the multi-source time accuracy, and generating a synchronized three-modal image group number set; S3: extracting the coordinates of the three modal centers from the images in the synchronous three-modal image group number set, comparing the deviations of the ultraviolet coordinates with those of the infrared image and the visible light image, and controlling the offset acquisition if the deviations exceed the sub-pixel registration tolerance to generate the alignment coordinates of the wire area image; S4: Based on the alignment coordinates of the conductor area image, the broken strand area and the length of the insulator spot chain are counted, the broken strand type is matched with the pollution level interval, and a broken strand type determination label and a pollution level identification number are generated; S5: Based on the broken strand type determination label and the contamination level identification number, the trimodal block coordinates are matched, a risk priority is assigned to each fusion area, and the contamination level numbers are sorted to form a cleaning priority list to generate multimodal fusion high-voltage equipment defect location data.

Images