A dual-light fusion intelligent online monitoring method based on video monitoring technology

Abstract

This invention relates to the field of computer vision technology, and more particularly to a dual-light fusion intelligent online monitoring method based on video monitoring technology. The dual-light fusion intelligent online monitoring method based on video monitoring technology includes the following steps: S1: Acquire visible light and infrared light images of a high-voltage disconnecting switch, and obtain the opening time period and closing time period based on the inter-frame differential segmentation of the video stream action sequence; obtain a status confirmation image based on the opening time period and the closing time period; S2: Extract the infrared light image from the status confirmation image and determine a first sequence, analyze the first sequence to obtain a temperature sequence before opening, a temperature sequence after opening, and a temperature change sequence. This invention constructs a dual-light fusion monitoring system, simultaneously extracts visible light mechanical trajectories and infrared temperature sequences, decouples fault influences, and generates a progressive defect identification model.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a dual-light fusion intelligent online monitoring method based on video monitoring technology. Background Technology

[0002] The opening and closing states of high-voltage disconnectors directly determine power grid safety, and their contact movement trajectories need to be accurately monitored to identify abnormal mechanical displacement or electrical contact faults. Existing video monitoring methods only statically analyze single-frame image features (such as contact overlap length or temperature value), lacking the ability to model the entire opening and closing process in a time sequence. This results in: the inability to correlate the mechanical displacement and temperature evolution sequence during the operation; difficulty in decoupling the dynamic coupling characteristics of mechanical / electrical faults; and the inability to detect gradual defects due to the lack of cross-time period feature comparison, leading to an increased false alarm rate.

[0003] Video monitoring technology, combined with dual-light fusion (visible light + infrared), can simultaneously capture changes in equipment's geometric displacement and temperature field, providing multi-dimensional evidence for fault diagnosis. While current dual-light technology can adapt to environmental interference such as changes in lighting, it fails to correlate the timing characteristics of opening and closing actions: the sudden temperature change after opening is caused by insufficient mechanical displacement, while closing misalignment exacerbates electrical contact overheating. Existing systems lack dynamic correlation analysis between the action process and the cause of the fault, making it difficult to pinpoint the root cause of complex faults.

[0004] Therefore, there is an urgent need to develop a dual-light fusion intelligent monitoring method: by collaboratively analyzing the mechanical trajectory and temperature evolution sequence during the opening and closing process, a fault cross-influence model can be established to achieve accurate separation and early warning of mechanical-electrical composite faults, thereby improving the reliability of power grid equipment condition diagnosis. Summary of the Invention

[0005] To overcome the drawback of undecoupled cross-effects of faults, this invention provides a dual-light fusion intelligent online monitoring method based on video monitoring technology.

[0006] The technical implementation scheme of the present invention is: a dual-light fusion intelligent online monitoring method based on video monitoring technology, comprising the following steps:

[0007] S1: Acquire visible light and infrared light images of the high-voltage disconnector, and obtain the opening and closing time periods based on the inter-frame differential segmentation of the video stream action sequence; obtain a status confirmation image based on the opening and closing time periods;

[0008] S2: Extract the infrared image based on the status confirmation image and determine the first sequence; analyze the first sequence to obtain the temperature sequence before tripping, the temperature sequence after tripping, and the temperature change sequence; determine the dominant fault based on the temperature sequence before tripping, the temperature sequence after tripping, and the temperature change sequence.

[0009] S3: Extract the visible light image based on the status confirmation image and determine the second sequence; analyze the second sequence to obtain the overlap length sequence and the offset angle sequence; determine the mechanical fault based on the overlap length sequence and the offset angle sequence;

[0010] S4: Based on the first sequence and the second sequence, construct a video keyframe feature descriptor resistant to illumination changes; based on the video keyframe feature descriptor, construct a support vector machine classification model to obtain the identification result of the progressive defects of the equipment; based on the identification result, determine the opening and closing status.

[0011] Preferably, the step of acquiring the visible light image and infrared light image of the high-voltage disconnector, and obtaining the opening time period and closing time period based on the inter-frame differential segmentation of the video stream action timing, includes: acquiring the initial state of the high-voltage disconnector; if it is in the closing state, the opening time period is obtained when a sudden increase in the proportion of moving pixels is detected, until the movement continuously weakens and ends in a silent state; if it is in the opening state, the closing time period is obtained when a sudden increase in the proportion of moving pixels is detected, until the movement returns to a silent state.

[0012] Preferably, obtaining the status confirmation image based on the opening time period and the closing time period includes:

[0013] During the opening and closing time periods, continuous frame images of the visible light image and the infrared light image are continuously acquired.

[0014] Calculate the pixel-by-pixel grayscale difference between the current frame and the previous frame to generate an inter-frame difference image;

[0015] A dynamic grayscale difference threshold is set, and pixels in the inter-frame difference image that exceed the dynamic grayscale difference threshold are marked as moving regions, while the rest are marked as stationary regions, thus forming a binary motion mask.

[0016] The proportion of pixels in the motion region to the total number of pixels in the binarized motion mask is statistically analyzed, and this proportion is recorded in real time to form a motion state sequence.

[0017] When the values ​​of the motion state sequence for multiple consecutive frames are lower than the preset static ratio threshold, it is determined that the opening or closing operation has ended, and the current frame is used as the status confirmation image.

[0018] Preferably, the step of extracting the infrared image based on the state confirmation image and determining a first sequence, and analyzing the first sequence to obtain a temperature sequence before circuit breaker tripping, a temperature sequence after circuit breaker tripping, and a temperature change sequence includes:

[0019] Before the start of the circuit breaker tripping period, the infrared light image is extracted and defined as the first infrared light image;

[0020] After the circuit breaker tripping period ends, the infrared light image is extracted and defined as the second infrared light image;

[0021] Similarly, the first infrared light image and the second infrared light image are extracted before and after N of the aforementioned time periods of circuit breaker opening;

[0022] The N first infrared light images and the second infrared light images are combined to form a sequence, which is defined as the first sequence;

[0023] Traverse the first sequence and perform the following operations on each group of images: calculate the highest temperature value of the contact area between the moving contact and the stationary contact in the first infrared light image, calculate the highest temperature value of the same area in the second infrared light image, and calculate the temperature change of the same area in the two images.

[0024] The highest temperature values ​​of all the first infrared images are counted to form a temperature sequence before the circuit breaker is opened. The highest temperature values ​​of all the second infrared images are counted to form a temperature sequence after the circuit breaker is opened. All the temperature changes are counted to form a temperature change sequence.

[0025] Preferably, determining the dominant fault based on the pre-trip temperature sequence, post-trip temperature sequence, and temperature change sequence includes:

[0026] Calculate the Pearson correlation coefficient between the temperature change sequence and the temperature sequence before the circuit breaker trips to obtain the first correlation.

[0027] Calculate the Spearman rank correlation coefficient between the temperature change sequence and the temperature sequence after the circuit breaker is opened to obtain the second correlation.

[0028] If the first correlation is greater than the preset threshold for electrical faults and the second correlation is less than the threshold for mechanical faults, then the electrical fault is determined to be dominant.

[0029] If the second correlation is greater than the mechanical fault threshold and the first correlation is less than the electrical fault preset threshold, then the mechanical fault is determined to be dominant.

[0030] If the first correlation is greater than the electrical fault preset threshold and the second correlation is greater than the mechanical fault threshold, then a mixed fault is determined.

[0031] If none of the above conditions are met, then there is no significant fault.

[0032] Preferably, the step of extracting the visible light image based on the state confirmation image and determining the second sequence, and analyzing the second sequence to obtain the overlap length sequence and the offset angle sequence, includes:

[0033] After the closing time period ends, the visible light image is extracted and defined as the visible light image after closing;

[0034] Similarly, the visible light images after closing are extracted after the N closing time periods have ended;

[0035] The N visible light images after the switch is closed are arranged into a sequence and defined as the second sequence;

[0036] Traverse the second sequence and perform the following operations for each group of images: measure the overlap length of the moving contact and the stationary contact in the visible light image after the circuit breaker is closed, and calculate the offset angle of the contact center axis in the visible light image after the circuit breaker is closed.

[0037] All the aforementioned overlap lengths are counted to form an overlap length sequence;

[0038] All the aforementioned offset angles are counted to form an offset angle sequence.

[0039] Preferably, determining the mechanical fault based on the overlap length sequence and the offset angle sequence includes:

[0040] Calculate the linear regression slope of the overlapping length sequence, perform wavelet transform on the overlapping length sequence, and detect step abrupt change points;

[0041] The variance sequence of the offset angle sequence is calculated within a sliding window, and outliers in the offset angle sequence are detected based on the three-standard-deviation principle.

[0042] If the linear regression slope is lower than the preset wear threshold, or the step abrupt change point is detected, or the variance sequence exceeds the set threshold, or the anomaly point is detected, then a mechanical fault is determined.

[0043] Preferably, the step of constructing a video keyframe feature descriptor resistant to illumination changes based on the first sequence and the second sequence includes:

[0044] Based on the analysis results of the first sequence and the second sequence, key regions of the visible light image and the infrared light image are extracted;

[0045] For the key region, calculate the local gradient direction distribution to construct feature descriptors for the device shape and texture;

[0046] The key region is divided into basic units, and the spatial regions of adjacent basic units are combined to form normalized blocks. The gradient direction histogram of each normalized block is calculated.

[0047] The gradient direction histograms connecting all normalized blocks are used to form a global description vector, and brightness normalization is performed on the global description vector.

[0048] Finally, a video keyframe feature descriptor resistant to illumination changes is constructed to identify progressive defect states caused by contact wear and mechanical deformation.

[0049] Preferably, the step of constructing a support vector machine classification model based on the video keyframe feature descriptors to obtain the identification result of progressive defects in the device includes:

[0050] The video keyframe feature descriptors are input into a pre-trained support vector machine classification model, and the optimal classification hyperplane in the support vector machine classification model is used to make a partitioning decision on the feature space.

[0051] The support vector machine classification model is trained based on historical samples, and its core mechanism is to find the hyperplane boundary that maximizes the classification margin.

[0052] When the feature expression is located in the positive side region of the hyperplane, the output is a normal state; when the feature expression is located in the negative side region of the hyperplane, the output is a contact wear or mechanical deformation state, and finally the identification result of the progressive defect of the equipment is obtained.

[0053] Preferably, determining the opening / closing status based on the identification result includes:

[0054] For the positive sample images output by the support vector machine classification model, an edge detection algorithm is used to determine the outer contour boundary of the high-voltage disconnector arm;

[0055] The outer contour boundary is converted into geometric straight line features using shape feature extraction technology;

[0056] The included angle of the outer contour of the high-voltage arm is calculated based on the geometric straight line characteristics and defined as the opening and closing angle;

[0057] When the opening and closing angle is between the preset opening position judgment angle and the closing position judgment angle, it is determined that the opening and closing operation is not in place and an alarm signal is triggered.

[0058] When the opening and closing angle exceeds the range of the determination angle, the opening and closing status is confirmed to be normal.

[0059] Beneficial Effects: This invention constructs a dual-light fusion collaborative monitoring system to simultaneously extract visible light mechanical trajectories and infrared temperature evolution sequences during the opening and closing process of high-voltage disconnecting switches. It combines time-series correlation analysis to decouple the cross-influence of mechanical and electrical faults and generates a progressive defect identification model based on geometrical optical invariance characteristics. This method overcomes the limitations of traditional single-light monitoring's static diagnosis, achieving dynamic correlation between action and fault: it determines the positioning status in real time based on the opening and closing angle, triggering graded alarms for mechanical misalignment or contact overheating; simultaneously, it separates the dominant fault type through the statistical correlation between temperature change sequences and mechanical trajectories, and predicts equipment degradation trends based on progressive defect characteristics. When a mixed fault or incomplete opening and closing is detected, composite fault location and mechanical correction commands are simultaneously activated; if the progressive defect exceeds limits, maintenance intervention is initiated in advance. This dual-light collaborative decision-making mechanism breaks through the limitations of static analysis in traditional video surveillance and innovatively achieves: dynamic feature decoupling: accurately segmenting action time periods through inter-frame difference algorithm and establishing a temporal correlation model of mechanical trajectory-temperature evolution; multimodal feature fusion: constructing geometric-optical invariant feature representations of visible light and infrared; and enhanced video understanding: the support vector machine classification model is trained based on historical video samples to achieve cross-time period recognition of progressive defects. Attached Figure Description

[0060] Figure 1 This is a flowchart of the dual-light fusion intelligent online monitoring method based on video monitoring technology according to the present invention;

[0061] Figure 2 This is a schematic diagram illustrating the principle of the inter-frame difference method.

[0062] Figure 3 This is a schematic diagram of the original color image and its RGB components;

[0063] Figure 4 This is a diagram showing the effect of grayscale processing of the image;

[0064] Figure 5 This is a schematic diagram of image border noise;

[0065] Figure 6 Flowchart for HOG feature extraction in key regions;

[0066] Figure 7 A diagram illustrating the decision-making principle of the SVM classification model;

[0067] Figure 8 This is a schematic diagram of the tripping position determination angle;

[0068] Figure 9 This is a schematic diagram of the closing position determination angle;

[0069] Figure 10 Flowchart of the circuit breaker opening / closing status recognition algorithm;

[0070] Figure 11 A schematic diagram of the isolating switch is captured for the moving target tracking algorithm;

[0071] Figure 12 These are images of some positive samples;

[0072] Figure 13 These are images containing some negative samples;

[0073] Figure 14 This is an image showing the infrared imaging effect. Detailed Implementation

[0074] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0075] As can be seen from the background technology, existing methods analyze single mechanical or electrical faults in isolation, without decoupling the cross-influence between the two, and lack the temporal correlation between the action process and the fault, resulting in a high misjudgment rate of compound faults and serious missed detection of progressive defects.

[0076] This invention achieves decoupling of cross-influence of faults and prediction of progressive defects by using dual-light fusion collaborative analysis of the mechanical trajectory and temperature evolution sequence of the opening and closing process, significantly improving the accuracy of multi-source fault diagnosis and the reliability of early warning.

[0077] A dual-light fusion intelligent online monitoring method based on video monitoring technology, such as Figure 1 As shown, it includes the following steps:

[0078] Example 1: S1-1: Acquire visible light and infrared light images of the high-voltage disconnector, and obtain the opening time period and closing time period based on the inter-frame differential segmentation of the video stream action sequence; acquire the initial state of the high-voltage disconnector; if it is in the closing state, mark the start of opening when a sudden increase in the proportion of moving pixels is detected, until the movement continuously weakens to the end of the silent state, thus obtaining the opening time period; if it is in the opening state, mark the start of closing when a sudden increase in the proportion of moving pixels is detected, and end when the movement returns to the silent state, thus obtaining the closing time period.

[0079] It should be noted that a high-definition infrared temperature measurement camera is used to simultaneously acquire visible light images (capturing mechanical displacement) and infrared images (monitoring the temperature field) of the high-voltage disconnector.

[0080] Example 2: S1-2: Obtain a status confirmation image based on the opening time period and the closing time period;

[0081] It should be noted that, since the opening and closing of the monitored high-voltage disconnecting switch is a dynamic process, a motion detection algorithm is chosen to monitor the disconnecting switch. Feature extraction and recognition technology of camera monitoring images is crucial for switch status diagnosis. Currently, the technology for identifying and analyzing images with prominent feature values ​​in static images is relatively mature, such as digit recognition and color recognition; however, the extraction, analysis, and processing of dynamic features in dynamic images still present technical challenges. The inter-frame difference method compares two consecutive frames to extract information about moving targets, and it has good adaptability to dynamic environments. The principle of the inter-frame difference method is as follows: Figure 2 As shown. Suppose the given video image sequence is f1, f2, ..., fn, where , For any two consecutive frames, the differenced image is described by the following formula: , Indicates the first The background model of the frame in coordinates Pixel value at that location, Indicates the first Frame in coordinates The pixel grayscale value at the given location is used to calculate the pixel-level absolute difference between the current frame and the background model, generating a motion region detection mask; dynamic background modeling ( It can adapt to changes in lighting and interference from swaying leaves, making it more robust than simple inter-frame difference; output For subsequent binarization (moving / static region segmentation), the background model is dynamically updated using a Gaussian mixture model. .

[0082] This invention requires uploading a frame of on-site image after the disconnecting switch is opened or closed. It compares frames from different times and uses an inter-frame difference method to determine faults in the monitored images. Specifically, based on the image data from consecutive time periods in the video surveillance image, the approximate shape and location information of the target are obtained through difference calculations. When an anomaly occurs in the video surveillance image, there will be a clear difference between frames. Subtracting the previous frame from the next frame yields the relative difference between frames, which is used to determine the current state of the image and the movement of the target.

[0083] A1: During the opening and closing time periods, continuously acquire consecutive frame images of the visible light image and the infrared light image;

[0084] A2: Calculate the pixel-by-pixel grayscale difference between the current frame and the previous frame to generate an inter-frame difference image;

[0085] It's important to note that an RGB image is an array of color pixels of size M×N×3. An RGB image is a "set" of three grayscale images. When sent to the red, green, and blue inputs of a color monitor, it produces a color image on the screen. Typically, the three images that make up an RGB color image are referred to as the Red, Green, and Blue component images. Figure 3 The image in the middle represents the original image and the extracted R, G, and B components.

[0086] When it's necessary to speed up processing while reducing storage space, image grayscale conversion is required. Image grayscale conversion refers to the process of converting an image from an RGB image to a grayscale image. A grayscale image contains only brightness information; a grayscale image is an array of size M×N×1, where each point is divided into 256 levels from 0 to 255, where 0 represents the darkest (pure black) and 255 represents the brightest (pure white). The following methods are commonly used to convert RGB images to grayscale: , To output grayscale values ​​(range 0-255), R, G, B: the red, green, and blue component values ​​(range 0-255) of the input RGB image at coordinates (M, N). Through this linear transformation, a grayscale image is obtained.

[0087] Figure 4 The image grayscale result is shown. It should be noted that, in order to reduce the interference of border information, the text part at the bottom of the image and the colorimetric table part on the right were manually removed before grayscale conversion.

[0088] Image noise exhibits both random and regular characteristics. The main causes of random noise include: internal noise generated by the random motion of particles within sensors or electronic components; noise caused by changes in current or electromagnetic fields due to mechanical vibrations of internal electrical components; noise from environmental interference; and borders and text added to the image by the camera manufacturer. The largest source of interference noise in monitoring images comes from white borders within the image, such as... Figure 5 As shown in the grayscale image, the border interference resembles a small "mutation".

[0089] In image processing, the method of eliminating noise by utilizing its properties is generally called image smoothing. The purpose of image smoothing is to eliminate or reduce noise interference in an image and improve its quality. Assuming additive noise is randomly and independently distributed, averaging or weighted averaging of neighborhood noise can effectively suppress noise interference. Assuming the noise is regular noise, median filtering can effectively filter out interference.

[0090] A3: Set a dynamic grayscale difference threshold, mark pixels in the inter-frame difference image that exceed the dynamic grayscale difference threshold as moving regions, and mark the rest as stationary regions to form a binary motion mask;

[0091] A4: Calculate the proportion of pixels in the motion region to the total pixels in the binary motion mask, and record this proportion in real time to form a motion state sequence;

[0092] A5: When the values ​​of the motion state sequence for multiple consecutive frames are lower than the preset static ratio threshold, it is determined that the opening or closing operation has ended, and the current frame is used as the status confirmation image.

[0093] It should be noted that continuous frame images of dual-light video are continuously acquired during the opening / closing time period (A1). Pixel-by-pixel grayscale value difference calculations are performed on adjacent frames to generate inter-frame difference images (A2). Moving regions (pixels exceeding the threshold) and stationary regions are segmented using a dynamic grayscale difference threshold, forming a binarized motion mask (A3). The percentage of pixels in the moving region is statistically analyzed in real time and recorded as a motion state sequence (A4). When the percentage of pixels in this sequence is below a preset stationary proportion threshold (e.g., 5%) for several consecutive frames, the operation is considered complete, and the current frame is used as the status confirmation image (A5). This process accurately captures the start and end points of mechanical movements through motion quantization, providing a static reference image for subsequent fault diagnosis.

[0094] Example 3: S2-1: Extract the infrared light image based on the status confirmation image and determine the first sequence; analyze the first sequence to obtain the temperature sequence before circuit breaker tripping, the temperature sequence after circuit breaker tripping, and the temperature change sequence;

[0095] B1: Before the start of the circuit breaker tripping period, the infrared light image is extracted and defined as the first infrared light image;

[0096] B2: After the circuit breaker tripping period ends, extract the infrared light image and define it as the second infrared light image;

[0097] B3: Similarly, extract the first infrared light image and the second infrared light image before and after N of the aforementioned time periods of circuit breaker opening;

[0098] B4: Form a sequence by combining N of the first infrared light images and the second infrared light images, and define it as the first sequence;

[0099] B5: Traverse the first sequence and perform the following operations on each group of images: calculate the highest temperature value of the contact area between the moving contact and the stationary contact in the first infrared light image, calculate the highest temperature value of the same area in the second infrared light image, and calculate the temperature change of the same area in the two images.

[0100] B6: Calculate the highest temperature value of all the first infrared light images to form a temperature sequence before the circuit breaker is opened, calculate the highest temperature value of all the second infrared light images to form a temperature sequence after the circuit breaker is opened, and calculate all the temperature changes to form a temperature change sequence.

[0101] It should be noted that infrared thermal imaging can clearly show the temperature distribution characteristics of the contact area of ​​the contact point, such as... Figure 14 As shown, traditional monitoring methods rely on the opening and closing motion curves to analyze faults. However, there is a strong coupling between the mechanical displacement and electrical contact of high-voltage disconnecting switches—mechanical jamming masks poor contact, while electrical overheating is easily misjudged as abnormal displacement. Although existing dual-light fusion technology can resist environmental interference, it still analyzes infrared temperature or visible light trajectories in isolation, failing to decouple the dynamic cross-influence of the two types of faults, resulting in single and unreliable video diagnostic conclusions.

[0102] For the tripping process, an infrared image of the contacts in close contact (first infrared image, representing the initial electrical state) is captured before the operation begins, and an infrared image of the separated state (second infrared image, reflecting residual temperature after the operation) is captured immediately after the operation. Paired images from multiple operations are constructed into a first sequence, and a temperature change sequence is generated by calculating the highest temperature difference in the contact area. This method accurately quantifies the electrical characteristics at the moment of tripping: normal tripping should be accompanied by a sudden temperature drop (complete contact separation); if the temperature change sequence is abnormal, it exposes residual high temperature due to an electrical fault; and the time-series comparison of the temperature sequences before and after tripping eliminates interference from insufficient mechanical displacement.

[0103] The spatiotemporal correlation between temperature evolution sequence and mechanical action provides a core criterion for fault decoupling in dual-light video, supporting accurate decision-making in subsequent state recognition models.

[0104] S2-2: Based on the temperature sequence before tripping, the temperature sequence after tripping, and the temperature change sequence, determine the dominant fault;

[0105] C1: Calculate the Pearson correlation coefficient between the temperature change sequence and the temperature sequence before the circuit breaker trips to obtain the first correlation.

[0106] C2: Calculate the Spearman rank correlation coefficient between the temperature change sequence and the temperature sequence after the circuit breaker is opened to obtain the second correlation.

[0107] C3: If the first correlation is greater than the preset threshold for electrical faults and the second correlation is less than the threshold for mechanical faults, then the electrical fault is determined to be dominant.

[0108] C4: If the second correlation is greater than the mechanical fault threshold and the first correlation is less than the electrical fault preset threshold, then the mechanical fault is determined to be dominant.

[0109] C5: If the first correlation is greater than the electrical fault preset threshold and the second correlation is greater than the mechanical fault threshold, then a mixed fault is determined.

[0110] C6: If none of the above conditions are met, then there is no significant fault.

[0111] It should be noted that the Pearson correlation coefficient is used to analyze the linear correlation between temperature change and the temperature sequence before tripping—the initial high temperature and temperature difference are strongly correlated during electrical faults (dominated by contact resistance); the Spearman rank correlation coefficient is used to detect the nonlinear trend between temperature change and temperature after tripping—the residual temperature and temperature difference show monotonically abnormalities when mechanical faults lead to incomplete separation. Electrical / mechanical fault thresholds are set based on historical fault sample statistics; for example, the electrical threshold is set to 0.8 (strong positive correlation), and the mechanical threshold is set to 0.7 (significant rank correlation, derived from historical fault sample statistics). The dual-correlation quantification model dynamically maps temperature evolution to fault type probability, decoupling the cross-interference between insufficient mechanical displacement and poor electrical contact, providing a physical basis for explaining the defect classification of video keyframes.

[0112] Example 4: S3-1: Extract the visible light image based on the state confirmation image and determine the second sequence; analyze the second sequence to obtain the overlap length sequence and the offset angle sequence;

[0113] D1: After the closing time period ends, extract the visible light image and define it as the visible light image after closing;

[0114] D2: Similarly, extract the visible light images after the closing time period ends for N closing time periods;

[0115] D3: Construct a sequence from the N visible light images obtained after the switch is closed, and define it as the second sequence;

[0116] D4: Traverse the second sequence and perform the following operations for each group of images: measure the overlap length of the moving contact and the stationary contact in the visible light image after the circuit is closed, and calculate the offset angle of the contact center axis in the visible light image after the circuit is closed.

[0117] D5: Count all the aforementioned overlap lengths to form an overlap length sequence;

[0118] D6: Count all the aforementioned offset angles to form an offset angle sequence.

[0119] It should be noted that traditional high-voltage disconnector monitoring only extracts static geometric parameters (such as contact overlap length) after a single closing operation. However, displacement anomalies caused by mechanical loosening or wear can only be captured through the temporal evolution of multiple operations. Existing video analysis, lacking comparison of continuous operation sequences, struggles to distinguish between instantaneous offsets and progressive mechanical faults.

[0120] This invention captures a visible light image after each closing operation, defined as a post-closing visible light image, which freezes the final contact state of the contacts. A second sequence is constructed by extracting these images from N operations. Dynamic analysis is performed by traversing the sequence: Overlap length sequence: quantifies the effective contact area of ​​the moving / stationary contacts; a continuously shortening length indicates contact wear; Offset angle sequence: measures the deviation angle of the contact center axis; increased angle fluctuations reflect loosening of the mechanical structure.

[0121] By tracking temporal geometric parameters, isolated images are upgraded into mechanical state evolution models. The linear decay of the overlap length and the abrupt change in the offset angle are directly related to specific mechanical faults (such as pin wear and connecting rod deformation), providing quantitative dynamic trajectory evidence for video diagnostics.

[0122] S3-2: Determine the mechanical fault based on the overlap length sequence and offset angle sequence;

[0123] E1: Calculate the linear regression slope of the overlapping length sequence, perform wavelet transform on the overlapping length sequence, and detect step abrupt change points;

[0124] E2: Calculate the variance sequence of the offset angle sequence within a sliding window, and detect outliers in the offset angle sequence based on the three-standard-deviation principle;

[0125] E3: If the linear regression slope is lower than the preset wear threshold, or the step abrupt change point is detected, or the variance sequence exceeds the set threshold, or the abnormal point is detected, then a mechanical fault is determined.

[0126] It should be noted that the slope of the linear regression of the overlap length sequence is lower than the preset wear threshold, indicating that the continuous deterioration of contact wear leads to a decrease in the effective contact area; wavelet transform is used to detect step abrupt change points and capture sudden mechanical damage caused by pin breakage. The increased variance of the offset angle sequence reflects trajectory instability caused by loosening of the mechanical structure. Identifying outliers based on the three-standard-deviation principle eliminates instantaneous interference and locks in the true displacement deviation.

[0127] By using a time-series statistical model, the geometric features in the dynamic video stream are quantified into mechanical health indicators, transforming discrete image frames into continuous fault evolution curves. When any of the above statistical anomalies is triggered, the video diagnostic system automatically associates them with specific mechanical fault modes (such as wear and deformation), achieving a closed loop from visual analysis to fault decision-making.

[0128] Example 5: S4-1: Construct a video keyframe feature descriptor resistant to illumination changes based on the first sequence and the second sequence;

[0129] It should be noted that before proceeding to Example 5, image borders need to be denoised. The actual transmitted images contain very little salt-and-pepper noise and Gaussian noise; the main noise comes from border interference added by the camera manufacturer. This invention uses a 7×7 median filter to smooth the infrared image.

[0130] F1: Based on the analysis results of the first sequence and the second sequence, extract the key regions of the visible light image and the infrared light image;

[0131] F2: For the key region, calculate the local gradient direction distribution to construct a feature descriptor for the device shape and texture;

[0132] F3: Divide the key region into basic units, combine the spatial regions of adjacent basic units to form normalized blocks, and calculate the gradient direction histogram of each normalized block;

[0133] F4: Connect the gradient direction histograms of all normalized blocks to form a global description vector, and perform brightness normalization on the global description vector;

[0134] F5: Finally, construct a video keyframe feature descriptor that is resistant to changes in lighting conditions, used to identify progressive defect states caused by contact wear and mechanical deformation.

[0135] It should be noted that the reference Figure 6 This scheme, by fusing the analysis results of visible light and infrared sequences (temperature sequence of Example 3 and geometric sequence of Example 4) with real-time acquired visible light and infrared images, locates the key areas of the isolating switch contacts. For these areas, a gradient direction histogram feature descriptor (i.e., HOG feature) is used to construct a feature representation resistant to illumination changes: Gradient statistics: Calculate the gradient direction of each pixel within the key area (reflecting edges and texture), and quantize the direction into discrete values ​​(e.g., 12 directions). Spatial segmentation: Divide the key area into basic units (e.g., 8×8 pixels), and combine adjacent units to form normalized blocks (e.g., 16×16 pixels). Statistically calculate the histogram of gradient directions within each block to form local features. Global description and normalization: Connect the histograms of all blocks to generate a global vector, and perform brightness normalization (formula: ,in, Represents the vector to be normalized. express The normalized norm, (represented by an extremely small constant) to eliminate the influence of differences in illumination intensity. This method combines key region information extracted from time-series analysis with the geometric-optical invariance of HOG, enabling stable characterization of gradual shape changes caused by contact wear or mechanical deformation.

[0136] Key areas: Examples 3 (temperature sequence) and 4 (overlap length / offset angle sequence)

[0137] The analysis identifies the contact area of ​​the contactor and the area of ​​the mechanical moving parts, as well as real-time acquired visible light and infrared light images. Normalization block: A spatial region composed of adjacent basic units (e.g., four 8×8 units), used to statistically analyze the local gradient direction histogram and normalize the brightness. Brightness normalization processing: The global description vector is scaled to ensure its magnitude is uniformly 1 (by adding a minimal constant ε to prevent division by zero), suppressing interference from changes in illumination. Progressive defect state: Equipment deterioration caused by long-term contact wear or slow deformation of the mechanical structure requires cross-time period feature comparison for identification.

[0138] For example, suppose the contact of a disconnector switch has changed edge texture due to wear: the key area of ​​the contact is located by the offset angle sequence of Example 4; the image of this area is extracted, basic units are divided and gradient directions are calculated (the gradient direction distribution at the wear point is abnormal); the units are combined into blocks, the gradient histograms of each block are statistically analyzed, and descriptive vectors are generated after connection; brightness normalization is used to eliminate the difference between morning and evening lighting, so that the features of the same wear state are consistent in images at different time periods; the support vector machine model identifies the progressive defects of contact wear accordingly.

[0139] Example 6: S4-2: Based on the video keyframe feature descriptors, construct a support vector machine classification model to obtain the identification results of progressive defects in the device;

[0140] G1: Input the video keyframe feature descriptors into a pre-trained support vector machine classification model, and make a partitioning decision on the feature space through the optimal classification hyperplane in the support vector machine classification model;

[0141] G2: The support vector machine classification model is trained based on historical samples. Its core mechanism is to find the hyperplane boundary that maximizes the classification margin.

[0142] G3: When the feature expression is located in the positive side region of the hyperplane, the normal state is output; when the feature expression is located in the negative side region of the hyperplane, the contact wear or mechanical deformation state is output, and finally the identification result of the progressive defect of the equipment is obtained.

[0143] It should be noted that the reference Figure 7As shown, the core mechanism of the Support Vector Machine (SVM) classification model is to find the optimal classification hyperplane to achieve binary classification decisions. During training based on historical samples, the model determines the boundary that maximizes the classification margin by solving an optimization problem that minimizes the square of the magnitude of the hyperplane's normal vector. Specifically, the labeled training samples are mapped to the feature space, the weight parameters are solved using the Lagrangian function, and finally, a decision function is generated. When a newly input feature descriptor lies on the positive side of the hyperplane, a positive class is output; when it lies on the negative side, a negative class is output. Mathematically, this is achieved by determining the classification result through a sign function.

[0144] This solution transforms the identification of progressive equipment defects into a binary classification problem: Model training: A support vector machine is trained using anti-lighting feature descriptors of historical normal and defective states (contact wear, mechanical deformation) to generate the optimal classification hyperplane boundary. Online identification: The real-time feature descriptors constructed in Example 5 are input into the pre-trained model, and a decision is made based on their spatial relationship with the hyperplane: features located on the positive side are output as normal states, and features located on the negative side are output as defective states. This method enhances generalization ability by maximizing the classification margin, effectively resisting changes in substation lighting and viewing angle interference, and achieving stable identification of progressive defects.

[0145] Example 7: S4-3: Determine the opening and closing status based on the identification results.

[0146] H1: For the positive sample images output by the support vector machine classification model, the outer contour boundary of the high-voltage disconnector arm is determined by the edge detection algorithm;

[0147] H2: The outer contour boundary is converted into geometric straight line features using shape feature extraction technology;

[0148] H3: Calculate the included angle of the outer contour of the high-voltage arm based on the geometric straight line characteristics, and define it as the opening and closing angle;

[0149] H4: When the opening and closing angle is between the preset opening position judgment angle and the closing position judgment angle, the opening and closing operation is determined to be incomplete and an alarm signal is triggered.

[0150] H5: When the opening and closing angle exceeds the range of the judgment angle, the opening and closing status is confirmed to be normal.

[0151] It should be noted that this step accurately determines the opening and closing status through geometric features:

[0152] Edge detection: The Canny algorithm is used to process the positive sample images output by the support vector machine. Gaussian filtering for noise reduction, gradient calculation, non-maximum suppression, and double thresholding are used to extract the outer contour boundary of the high-voltage arm, ensuring accurate edge localization and noise resistance. Line transformation: The Hough transform is used to map the contour boundary to the parameter space, detect straight line features, and fit the geometric axis of the high-voltage arm. Angle calculation: The angle of the outer contour of the high-voltage arm is calculated based on the geometric relationship of the axis, defined as the opening and closing angle.

[0153] Figure 8 (a) and Figure 9 (a) shows a set of original video frames of the opening and closing of a disconnecting switch, identified using HOG and SVM algorithms. The high-voltage arm boundary contour was then extracted using the Canny algorithm (an edge detection algorithm), and after Hough transform processing, a geometrically distinct high-voltage arm contour curve was obtained, as shown below. Figure 8 (b) and Figure 9 As shown in (b). The straight-line shape of the outer contour of the high-voltage disconnector arm is accurately identified from the figure. This invention defines the included angle of the outer contour of the high-voltage arm as the "opening / closing angle," while the opening / closing angles of the high-voltage disconnector in its normal opening and closing states are respectively called the "opening position judgment angle" and the "closing position judgment angle." If the angle value of the "opening / closing angle" is between the "opening position judgment angle" and the "closing position judgment angle," then the high-voltage disconnector is determined to be in an incomplete opening / closing state.

[0154] The algorithm flow for identifying the opening and closing status of the circuit breaker is as follows: Figure 10 The substation inspection robot, equipped with a high-definition camera, captures real-time images of the substation, searches for original images of specific models of high-voltage disconnect switches, and after HOG feature extraction and SVM algorithm classification, determines whether the image is a positive sample (i.e., the image contains the specific model of high-voltage disconnect switch) or a negative sample (i.e., the image does not contain the specific model of high-voltage disconnect switch). If it is a negative sample, the calculation ends; if it is a positive sample, it determines whether the opening / closing angle (N) is between the preset "closing position judgment angle (N1)" and "opening position judgment angle (N2)". If it is not between the two angles, it is determined that "the opening / closing is in place"; if it is between the two angles, it is determined that "the opening / closing is not in place", and an alarm signal is issued.

[0155] The determination of the opening and closing status relies on preset angle thresholds: Normal state: When the opening angle is greater than the opening completion judgment angle, or the closing angle is less than the closing completion judgment angle, the status is confirmed to be normal. Abnormal alarm: If the opening and closing angles are between the two judgment angles, it indicates that the high-voltage arm has not been fully opened or closed, which is judged as an operation error and triggers an alarm. This method converts image edge information into quantifiable mechanical angles, and combined with preset thresholds, achieves objective diagnosis of the opening and closing status, effectively avoiding misjudgments caused by lighting or viewing angle interference.

[0156] Example 7: Due to the presence of numerous static equipment interferences (such as transmission lines and high-voltage towers) at substation sites, direct status identification can easily lead to misjudgments. This example uses a moving target detection algorithm to separate the dynamic operating components of the disconnector switch from the video stream, eliminating background interference to improve recognition accuracy. The specific steps are as follows: Step 1: Moving target detection (dynamic region segmentation), Input: Dual-light video stream (visible light + infrared). Processing: Use a Gaussian mixture model to extract the moving foreground (such as the opening and closing action area of ​​the disconnector switch). Fill the gaps through morphological operations to generate a connected moving target mask (based on a moving target tracking algorithm to extract the disconnector switch, such as...). Figure 11 (Example). Extract the minimum bounding rectangle of the target and output the moving target image fragment (focusing on the key components of the disconnect switch). Output: Segmented moving target region image (reducing background interference). Step 2: Target recognition (state classification and defect detection), input: image fragment output from moving target detection. Processing: Sample training: Positive samples: GW4 / GW6 disconnect switch image (real shot at a distance of 10m). Figure 12 Negative samples: background of transmission lines and high-voltage towers ( Figure 13 Preprocessing: 7×7 median filtering removes bounding box noise, and morphological processing enhances features. Feature extraction: HOG features (Local Gradient Direction Statistics) are calculated within the moving target region. Gradient histograms of normalized blocks are concatenated to generate feature descriptors resistant to illumination variations. SVM classification decision: Input feature vectors to a pre-trained SVM model (GPU-accelerated training). Output classification results: Normal state, contact wear, or mechanical deformation. Output: Isolating switch status label and defect type.

[0157] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A dual-light fusion intelligent online monitoring method based on video monitoring technology, characterized in that, Includes the following steps: S1: Acquire visible light and infrared light images of the high-voltage disconnector, and obtain the opening and closing time periods based on the inter-frame differential segmentation of the video stream action sequence; obtain a status confirmation image based on the opening and closing time periods, including: During the opening and closing time periods, continuous frame images of the visible light image and the infrared light image are continuously acquired. Calculate the pixel-by-pixel grayscale difference between the current frame and the previous frame to generate an inter-frame difference image; A dynamic grayscale difference threshold is set, and pixels in the inter-frame difference image that exceed the dynamic grayscale difference threshold are marked as moving regions, while the rest are marked as stationary regions, thus forming a binary motion mask. The proportion of pixels in the motion region to the total number of pixels in the binarized motion mask is statistically analyzed, and this proportion is recorded in real time to form a motion state sequence. When the values ​​of the motion state sequence are lower than a preset static ratio threshold for multiple consecutive frames, the opening or closing operation is determined to be completed, and the current frame is used as the status confirmation image; S2: Extract the infrared image based on the status confirmation image and determine the first sequence; analyze the first sequence to obtain the temperature sequence before tripping, the temperature sequence after tripping, and the temperature change sequence; determine the dominant fault based on the temperature sequence before tripping, the temperature sequence after tripping, and the temperature change sequence, including: Before the start of the circuit breaker tripping period, the infrared light image is extracted and defined as the first infrared light image; After the circuit breaker tripping period ends, the infrared light image is extracted and defined as the second infrared light image; Similarly, the first infrared light image and the second infrared light image are extracted before and after N of the aforementioned time periods of circuit breaker opening; The N first infrared light images and the second infrared light images are combined to form a sequence, which is defined as the first sequence; Traverse the first sequence and perform the following operations on each group of images: calculate the highest temperature value of the contact area between the moving contact and the stationary contact in the first infrared light image, calculate the highest temperature value of the same area in the second infrared light image, and calculate the temperature change of the same area in the two images. The highest temperature values ​​of all the first infrared images are counted to form a temperature sequence before the circuit breaker is tripped; the highest temperature values ​​of all the second infrared images are counted to form a temperature sequence after the circuit breaker is tripped; and all the temperature changes are counted to form a temperature change sequence. S3: Extract the visible light image based on the status confirmation image and determine the second sequence; analyze the second sequence to obtain an overlap length sequence and an offset angle sequence; determine the mechanical fault based on the overlap length sequence and the offset angle sequence, including: After the closing time period ends, the visible light image is extracted and defined as the visible light image after closing; Similarly, the visible light images after closing are extracted after the N closing time periods have ended; The N visible light images after the switch is closed are arranged into a sequence and defined as the second sequence; Traverse the second sequence and perform the following operations for each group of images: measure the overlap length of the moving contact and the stationary contact in the visible light image after the circuit breaker is closed, and calculate the offset angle of the contact center axis in the visible light image after the circuit breaker is closed. All the aforementioned overlap lengths are counted to form an overlap length sequence; All the aforementioned offset angles are counted to form an offset angle sequence; S4: Based on the first sequence and the second sequence, construct a video keyframe feature descriptor resistant to illumination changes; based on the video keyframe feature descriptor, construct a support vector machine classification model to obtain the identification result of the progressive defects of the equipment; based on the identification result, determine the opening and closing status.

2. The dual-light fusion intelligent online monitoring method based on video monitoring technology according to claim 1, characterized in that, The process of acquiring visible light and infrared light images of the high-voltage disconnector and obtaining the opening and closing time periods based on the inter-frame differential segmentation of the video stream's action timing includes: acquiring the initial state of the high-voltage disconnector; if it is in the closing state, marking the start of opening when a sudden increase in the proportion of moving pixels is detected, and continuing until the movement continuously weakens to a silent state to obtain the opening time period; if it is in the opening state, marking the start of closing when a sudden increase in the proportion of moving pixels is detected, and ending when the movement returns to a silent state to obtain the closing time period.

3. The dual-light fusion intelligent online monitoring method based on video monitoring technology according to claim 1, characterized in that, The process of determining the dominant fault based on the pre-trip temperature sequence, post-trip temperature sequence, and temperature change sequence includes: Calculate the Pearson correlation coefficient between the temperature change sequence and the temperature sequence before the circuit breaker trips to obtain the first correlation. Calculate the Spearman rank correlation coefficient between the temperature change sequence and the temperature sequence after the circuit breaker is opened to obtain the second correlation. If the first correlation is greater than the preset threshold for electrical faults and the second correlation is less than the threshold for mechanical faults, then the electrical fault is determined to be dominant. If the second correlation is greater than the mechanical fault threshold and the first correlation is less than the electrical fault preset threshold, then the mechanical fault is determined to be dominant. If the first correlation is greater than the electrical fault preset threshold and the second correlation is greater than the mechanical fault threshold, then a mixed fault is determined. If none of the above conditions are met, then there is no significant fault.

4. The dual-light fusion intelligent online monitoring method based on video monitoring technology according to claim 1, characterized in that, The step of determining mechanical faults based on the overlap length sequence and the offset angle sequence includes: Calculate the linear regression slope of the overlapping length sequence, perform wavelet transform on the overlapping length sequence, and detect step abrupt change points; The variance sequence of the offset angle sequence is calculated within a sliding window, and outliers in the offset angle sequence are detected based on the three-standard-deviation principle. If the linear regression slope is lower than the preset wear threshold, or the step abrupt change point is detected, or the variance sequence exceeds the set threshold, or the anomaly point is detected, then a mechanical fault is determined.

5. The dual-light fusion intelligent online monitoring method based on video monitoring technology according to claim 1, characterized in that, The step of constructing a video keyframe feature descriptor resistant to illumination changes based on the first sequence and the second sequence includes: Based on the analysis results of the first sequence and the second sequence, key regions of the visible light image and the infrared light image are extracted; For the key region, calculate the local gradient direction distribution to construct feature descriptors for the device shape and texture; The key region is divided into basic units, and the spatial regions of adjacent basic units are combined to form normalized blocks. The gradient direction histogram of each normalized block is calculated. The gradient direction histograms connecting all normalized blocks are used to form a global description vector, and brightness normalization is performed on the global description vector. Finally, a video keyframe feature descriptor resistant to illumination changes is constructed to identify progressive defect states caused by contact wear and mechanical deformation.

6. The dual-light fusion intelligent online monitoring method based on video monitoring technology according to claim 1, characterized in that, The step of constructing a support vector machine classification model based on the video keyframe feature descriptors to obtain the identification results of progressive defects in the device includes: The video keyframe feature descriptors are input into a pre-trained support vector machine classification model, and the optimal classification hyperplane in the support vector machine classification model is used to make a partitioning decision on the feature space. The support vector machine classification model is trained based on historical samples, and its core mechanism is to find the hyperplane boundary that maximizes the classification margin. When the feature expression is located in the positive side region of the hyperplane, the output is a normal state; when the feature expression is located in the negative side region of the hyperplane, the output is a contact wear or mechanical deformation state, and finally the identification result of the progressive defect of the equipment is obtained.

7. The dual-light fusion intelligent online monitoring method based on video monitoring technology according to claim 1, characterized in that, Determining the opening / closing status based on the identification result includes: For the positive sample images output by the support vector machine classification model, an edge detection algorithm is used to determine the outer contour boundary of the high-voltage disconnector arm; The outer contour boundary is converted into geometric straight line features using shape feature extraction technology; The included angle of the outer contour of the high-voltage arm is calculated based on the geometric straight line characteristics and defined as the opening and closing angle; When the opening and closing angle is between the preset opening position judgment angle and the closing position judgment angle, it is determined that the opening and closing operation is not in place and an alarm signal is triggered. When the opening and closing angle exceeds the range of the determination angle, the opening and closing status is confirmed to be normal.

Images