Inspection defect detection and analysis method for traction substation

Abstract

This invention belongs to the field of remote monitoring technology, specifically relating to a method for detecting and analyzing defects during inspections of traction substations. The method includes the following steps: Step 1, data acquisition and target area determination: Deploying image sensing devices with rolling exposure capabilities at fixed locations within the traction substation, and reading the row reading sequence number of each frame from the image sensing device as a row-level time index; Step 2, core processing and judgment symbol generation: Aligning physical modulation labels, geometric conclusions, and color-coded valid indicator lights to generate judgment symbols; Step 3, WFST state machine analysis and event output: Constructing a weighted finite state transition machine (WFST), and publishing event categories via voice broadcast and platform push. This invention significantly reduces the intensity of manual analysis and the error rate, maintaining high stability and reliability even under complex lighting, multiple interferences, and alternating flashing of multiple lights, demonstrating good engineering application value and promising prospects for widespread adoption.

Description

Technical Field

[0001] This invention belongs to the field of remote monitoring technology, specifically relating to a method for detecting and analyzing defects in traction substations. Background Technology

[0002] In the field of automated inspection and safety monitoring of traction substations, recent technological advancements have primarily focused on video image-based status recognition and intelligent diagnosis. Traditional traction substation inspections typically rely on manual patrols or semi-automated image acquisition systems. Personnel judge the color and flashing status of indicator lights by monitoring images or periodically taking photographs. While simple to operate, this method is heavily influenced by subjective experience and prone to misjudgments under conditions of strong light, backlighting, reflections, and camera angle deviations. For example, during the day under direct sunlight or at night under strong reflections, red, yellow, and green indicator lights may be obscured by light saturation in the image, leading to distorted color recognition results. At night in low-light conditions, the indicator lights may be too dim to be effectively distinguished from the background.

[0003] With the development of image processing and machine learning technologies, some studies have attempted to use methods such as color thresholding, template matching, and convolutional neural networks to achieve indicator light recognition. These methods achieve high recognition accuracy under ideal lighting conditions, but still have significant limitations in complex lighting environments. Color thresholding relies on fixed color ranges; once the environmental spectrum changes, the threshold becomes ineffective, easily leading to red-green confusion or misidentification. Template matching typically requires a fixed camera installation and a constant viewing angle; if the equipment vibrates or experiences slight shifts, the matched template deviates from the actual area, reducing detection accuracy. While deep learning methods based on convolutional neural networks have some adaptive capabilities, they require a large amount of labeled data for training, which is difficult to achieve in high-safety scenarios like substations where collecting diverse sample data is challenging. Furthermore, deep models consume significant computational resources, making long-term real-time operation on edge devices difficult. Summary of the Invention

[0004] Therefore, the main objective of this invention is to provide a method for defect detection and analysis during traction substation inspections. This method accurately separates the actual flicker signal through row-level time-indexed rolling shutter frequency demodulation, ensures morphological consistency using a-contrario geometric verification, achieves steady-state color recognition under strong light and backlighting by combining quaternion color invariants, and accurately outputs normal indications, operation prompts, or abnormal events through WFST state machine syntax parsing of multi-frame judgment symbols. Compared with traditional inspection methods, this invention effectively improves the automation level and recognition accuracy of substation inspections, significantly reduces the intensity of manual analysis and the error rate, and maintains high stability and reliability even under complex lighting, multiple interferences, and alternating flickering conditions of multiple lights, demonstrating good engineering application value and promising prospects for widespread adoption.

[0005] The technical solution adopted in this invention is as follows:

[0006] The method for detecting and analyzing defects during traction substation inspections includes the following steps:

[0007] Step 1, Data Acquisition and Target Area of ​​Interest Determination: Deploy image sensing devices with rolling exposure at fixed locations in the traction substation, set a fixed focal length and fixed exposure, and continuously acquire no less than 12 frames of color images; in the first frame, mark the target area of ​​interest on the indicator panel in a polygonal manner, record the pixel coordinates of the polygon vertices, and crop the target area of ​​interest sub-images in each frame image according to the recorded pixel coordinates of the polygon vertices; read the row reading sequence number of each frame from the image sensing device as a row-level time index;

[0008] Step 2, Core Processing and Judgment Symbol Generation: Within the target interest region sub-image of each frame, perform rolling shutter frequency demodulation to generate physical demodulation labels, and perform quaternion color invariant generation to identify valid color indicator lights, and then perform a-contrario geometric test to generate geometric conclusions; align the physical demodulation labels, geometric conclusions and valid color indicator lights to generate judgment symbols.

[0009] Step 3, WFST State Machine Parsing and Event Output: Construct a weighted finite state transition machine (WFST). Take the decision symbols generated in Step 2 as input and parse them using the WFST state machine to output event categories, where the event category is one of normal indication, operation prompt, or abnormal event. After parsing, overlay the target interest region outline, main color label, stripe band, and physical peak position on the corresponding frame to form an evidence map, and publish the event category via voice broadcast and platform push.

[0010] Furthermore, the rolling shutter frequency demodulation includes: converting the target region of interest sub-image into a brightness map, and performing a top-hat operation with a 3×3 structuring element to obtain a highlight enhancement map; calculating the difference map of adjacent rows in the row direction of the highlight enhancement map, the difference method being to subtract the (n+1)th row from the nth row to obtain a difference map that only reflects the brightness change in the row direction; traversing the difference map column by column, recording the bright-to-dark boundary from positive to negative and the dark-to-bright boundary from negative to positive, and alternately pairing the two types of boundaries to form bands; merging bands with a height of less than 2 pixels with adjacent bands of the same type to obtain a set of stable bands; and performing connected component aggregation on the adjacent column directions of the stable bands to obtain... To identify the stripe bands; locate the row number of the first dark-to-light boundary appearing from top to bottom in each column as the stripe phase start point for that column; generate physical demodulation labels according to the following rules based on the number of stripe bands and the row number offset of the stripe phase start point in adjacent frames: when the number of stripe bands is greater than or equal to 3 and the stripe phase start point moves monotonically downward or monotonically upward in adjacent frames, it is recorded as flickering; when the number of stripe bands is equal to 0 and the proportion of bright pixels in the target area of ​​interest sub-image is greater than or equal to 90%, it is recorded as constant brightness; when the number of stripe bands is equal to 0 and the proportion of bright pixels is less than or equal to 10%, it is recorded as constant darkness; the rest are recorded as weak flickering.

[0011] Further, the a-contrario geometric verification includes: calculating the Sobel amplitude map of the target region of interest sub-image, and obtaining the edge map through non-maximum suppression and double threshold connection; calculating the proportion of edge pixels in the target region of interest sub-image as the edge density baseline; generating two sets of concentric sampling rings with radii of 1 and 2 times the diameter of the indicator light at the center of the connected component of the color effective indicator light generated in step 2; sampling 36 points at equal intervals on each set of concentric sampling rings, and counting the hit counts falling on the edge map; comparing the hit counts with the expected value of edge density baseline × 36; when the hit counts of both sets of concentric sampling rings are at least 8 more than the corresponding expected value, the corresponding connected component is confirmed as a geometrically effective indicator light; performing skeletonization on the stripe obtained by rolling shutter demodulation in step 2, along the skeleton... The frame samples points every 5 pixels to obtain a point sequence, and constructs a sliding window of length 7 sequentially. Within each sliding window, it calculates a sequence of signs for the angles between 5 adjacent directions and counts the turning changes. When the turning change count of any sliding window is less than or equal to 1, the stripe is marked as a straight stripe. When the number of straight stripes in a frame is greater than or equal to 3, the frame's stripe structure is confirmed to be geometrically consistent. Based on the presence of the geometrically valid indicator light and whether the stripe structure is geometrically consistent, a geometric conclusion is output, where: when the geometrically valid indicator light is present and the stripe structure is geometrically consistent, a geometrically passed conclusion is output; when the geometrically valid indicator light is present but the condition for geometrically consistent stripe structure is not triggered, only the indicator light is passed conclusion; when the condition for geometrically consistent stripe structure is triggered but the geometrically valid indicator light has not yet appeared, only the stripe is passed conclusion.

[0012] Furthermore, the quaternion color invariant generation includes: calculating the channel averages of the red, green, and blue channels; using the average of the three channel averages as a reference value; performing gain correction on each channel by dividing the reference value by the channel average value to obtain the corrected red, green, and blue channels; using the arithmetic mean of the corrected red, green, and blue channels as the luminance, and assembling quaternion components in the order of luminance, red, green, and blue; performing conjugation operations on the quaternion components and calculating the square root of the sum of the squares of the four components as the joint intensity; dividing each of the four components by the joint intensity to obtain the normalized quaternion components; taking the red, green, and blue components of the normalized quaternion components as the color invariant map; identifying saturated pixels using the maximum output value of the image sensor, performing a 3×3 closing operation to expand the saturated region, and replacing the saturated region with the mean of the adjacent unsaturated pixels; marking connected components within the color invariant map, determining the primary color label based on the pixel counts of the red, green, and blue components in each connected component, and outputting the connected components with the primary color label as color valid indicator lights.

[0013] Furthermore, the rolling shutter heterodyne demodulation also includes: calculating the difference sequence by numbering the row of the fringe phase start point according to the frame number in the acquired image sequence; constructing an equivalent heterodyne envelope peak when the absolute value of the difference sequence remains the same in 3 consecutive frames or increases only by a step size of 1 pixel; and recording the object demodulation peak on the frame where the equivalent heterodyne envelope peak appears.

[0014] Furthermore, the generation of the determination symbol includes: within the same frame, matching the physics modulation label, geometric conclusion, and color valid indicator light according to pixel position overlap, using the overlapping pixel count of the connected component of the color valid indicator light and the stripe as the standard, and determining a successful match when the overlapping pixel count accounts for more than or equal to 30% of the connected component pixel count; when a successful match is achieved, generating a determination symbol based on the combination of the physics modulation label, the main color label of the color valid indicator light, and the geometric conclusion.

[0015] Furthermore, the determination of the primary color label includes: when the difference in pixel count between the red channel and the green channel is less than or equal to 10% of the total pixel count in the connected region, and the pixel counts of both the red channel and the green channel are greater than the pixel count of the blue channel, the primary color label is set to yellow.

[0016] Furthermore, the generation of the decision symbols also includes: when a match is successful and the physics demodulation label is flashing, the primary color label is red, green, or yellow, and the geometric conclusion is "geometric pass" or "only indicator light pass", outputting a decision symbol of red flashing, green flashing, or yellow flashing; when a match is successful and the physics demodulation label is constantly on, and the primary color label is red, green, or yellow, outputting a decision symbol of red constantly on, green constantly on, or yellow constantly on; when the physics demodulation label is constantly off and the color valid indicator light is not present, outputting a decision symbol of being constantly off; when the decision symbols of two adjacent frames change from green constantly on through green flashing or constantly off to red constantly on, outputting a decision symbol of green to red switching; when the decision symbols of two adjacent frames change from red constantly on through red flashing or constantly off to green constantly on, outputting a decision symbol of red to green switching.

[0017] Furthermore, the input alphabet of the WFST state machine is the set of decision symbols generated in step 2, and the output alphabet is the set of normal indications, operation prompts, and abnormal events. The state set includes: initial, green constant, red constant, yellow constant, green flashing, red flashing, yellow flashing, switching, alarm, and end.

[0018] Furthermore, the WFST state machine parsing includes the following state transition rules: Read the green flashing judgment symbol in the green constant-on state, transition to the green flashing state and output an operation prompt; read the red constant-on judgment symbol in the green constant-on state, transition to the alarm state and output an abnormal event; read the red-to-green switching judgment symbol in the red constant-on state, transition to the switching state and output an operation prompt; read the same color flashing judgment symbol three times consecutively in any flashing state, transition to the alarm state and output an abnormal event; read the same color constant-on judgment symbol twice consecutively in any flashing state, return to the corresponding constant-on state and output a normal indication; read any constant-on judgment symbol in the alarm state, transition to the corresponding constant-on state and output an abnormal event, then read any constant-on judgment symbol again, transition to the end state and output a normal indication.

[0019] By adopting the above technical solutions, this invention achieves the following beneficial effects: It maintains extremely high recognition stability under complex lighting and multi-interference environments. Through row-level time-indexed rolling shutter frequency demodulation, this invention achieves physical demodulation of indicator light radiation, enabling the system to capture changes in the flicker signal within the microsecond time domain. This accurately distinguishes between genuine flicker and reflective disturbances, effectively avoiding misjudgments caused by ambient light, background light sources, or camera equipment vibration. By introducing an a-contrario geometric verification mechanism, the spatial morphology of valid color indicator lights is statistically verified, maintaining stable detection even when the indicator light overlaps with the background edge, there is surface reflection, or interference from adjacent devices, fundamentally improving the robustness of geometric consistency determination. Combined with a quaternion color invariant color representation method, it eliminates brightness dependence under strong direct light or backlight conditions, allowing color recognition to be entirely determined by the normalized relative color components. This ensures that the red, yellow, and green states remain clearly distinguishable even under drastic lighting fluctuations, avoiding the drift problem of traditional RGB space algorithms under nonlinear lighting. Ultimately, this invention achieves syntax-level constraint parsing of decision symbols through a WFST state machine, enabling structured modeling of the temporal relationships between states such as flashing, constant illumination, and switching. This allows for accurate output of semantic results such as normal indications, operational prompts, or abnormal events within a continuous frame sequence. This structured parsing mechanism enables the system to automatically identify logical differences between operational and abnormal behaviors, reducing false alarm and missed alarm rates, and exhibiting stability and consistency, especially under conditions involving multiple light panels, periodic switching, and complex flashing signals. The overall solution of this invention, through multi-layered collaboration across physical, geometric, color, and semantic layers, significantly improves inspection accuracy and real-time performance, reduces the need for manual intervention and secondary verification, and provides a highly feasible, stable, and engineering-practical solution for intelligent inspection of traction substations. Attached Figure Description

[0020] Figure 1 A schematic diagram of the method flow for detecting and analyzing defects in traction substations provided in an embodiment of the present invention;

[0021] Figure 2 This is an experimental curve diagram of stripe phase shift detection provided in an embodiment of the present invention;

[0022] Figure 3 This is a schematic diagram of the a-contrario geometric inspection sampling loop provided in an embodiment of the present invention. Detailed Implementation

[0023] All features disclosed in this specification, or all steps in all disclosed methods or processes, may be combined in any way, except for mutually exclusive features and / or steps.

[0024] Any feature disclosed in this specification (including any appended claims and abstract) may be replaced by other equivalent or similar features, unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is merely one example of a series of equivalent or similar features.

[0025] Example 1: Reference Figure 1 The method for detecting and analyzing defects in traction substations includes the following steps:

[0026] Step 1, Data Acquisition and Target Area of ​​Interest Determination: Deploy an image sensing device with rolling exposure at a fixed location in the traction substation, set a fixed focal length and fixed exposure, and continuously acquire no less than 12 frames of color images; in the first frame, mark the target area of ​​interest on the indicator panel in a polygonal manner, record the pixel coordinates of the polygon vertices, and crop the target area of ​​interest sub-images in each frame image according to the recorded pixel coordinates of the polygon vertices; read the row reading sequence number of each frame from the image sensing device as a row-level time index.

[0027] In one specific implementation, when deploying an image sensing device with rolling exposure at a fixed location in a traction substation, the shooting direction is kept as close as possible to the normal direction of the indicator panel to reduce perspective distortion and ensure that the target area of ​​interest sub-images cropped according to the pixel coordinates of the polygon vertices in each subsequent image are geometrically consistent. The typical installation height is 2.5 to 3.5 meters above the ground, and the horizontal distance between the lens and the indicator panel is 4 to 8 meters, keeping the indicator panel's proportion in the image between 20% and 40% of the image width. This facilitates obtaining sufficient pixel detail while preserving surrounding references to aid in manual annotation. After installation, the image sensing device's autofocus, auto exposure, and auto white balance are turned off, and a fixed focal length and exposure are set; for example, a lens with an equivalent focal length of approximately 8 mm, an exposure time of approximately 1 / 500 second, an analog gain of approximately 6 dB, and a white balance fixed to the daylight preset. The reason for choosing a fixed focal length and fixed exposure is that, through stable optical imaging and stable exposure response, a visual input with consistent contrast and brightness can be obtained in each frame of the image. This avoids the introduction of uncertain brightness suppression or gain enhancement by automatic algorithms in strong light and backlight scenes, and provides a stable input from the source for subsequent rolling shutter frequency demodulation and quaternion color invariant generation.

[0028] Image acquisition employs a continuous acquisition method of at least 12 frames of color images. The frames per second (fps) can be set to 30, with an acquisition duration of approximately 0.4 seconds; alternatively, it can be set to 25 or 60 to adapt to the ambient lighting and indicator light blinking rhythm. The reason for choosing at least 12 frames is that the common indicator light blinking rhythm and brightness fluctuations caused by grid lighting do not strictly synchronize with the rolling exposure line-by-line reading. By acquiring multiple frames in a short period, sufficient line-level brightness change trajectories covering multiple fringe change cycles can be captured, providing sufficient temporal evidence for subsequent physical modulation. To avoid motion blur, the image sensor and the object being photographed are kept relatively stationary during acquisition; the bracket is fixed at three points with anti-loosening washers.

[0029] The pixel coordinate system for each frame takes the top-left corner of the image as the origin, with horizontal column coordinates increasing and vertical row coordinates increasing. The image resolution can be set to 1920×1080. When it is necessary to balance detail between strong light and backlight, the wide dynamic range (WDL) mode of the image sensor can be enabled, while maintaining a fixed exposure time and fixed gain to avoid introducing gain fluctuations between frames. The WDL mode can enhance the visible detail in both bright and shadow areas within the same frame, making the edge contours and markings on the indicator panel clearer when the target area of ​​interest is marked as a polygon in the first frame, facilitating accurate delineation of the polygon vertices' pixel coordinates.

[0030] On the first frame image, the target region of interest (PGI) of the indicator panel is marked using a polygon format via the user interface. Specifically, the first frame image is magnified to 200% to 300% on the monitor, and points are clicked along the visible hard or structural edges of the panel to form a polygon with 6 to 12 vertices. For rounded corners or curved edges, denser point sampling is used to fit the contour. After marking, the pixel coordinates of the polygon vertices are saved to persistent storage as paired integers of row and column coordinates. The marking time and image resolution used are also recorded to ensure that subsequent frames can reuse the same vertex pixel coordinates to crop the PPI sub-image. Polygons are used instead of rectangles because they can closely fit the actual panel boundaries, reducing the entry of irrelevant background pixels into the PPI sub-image and minimizing interference from subsequent color statistics and stripe detection. More than six vertices are used because more vertices can more accurately fit irregular boundaries, especially when backlighting reduces edge contrast, maintaining the fit of the cropped boundary.

[0031] Before cropping to obtain the target region of interest sub-image, lens distortion correction is performed on the first frame image using the lens parameter file obtained from the calibration board during the installation phase. A mapping table is saved after correction, and subsequent frames reuse the same table for rapid correction. This is because radial and tangential lens distortion can cause pixel-level offsets in the image edge regions, and the pixel coordinates of polygon vertices are inconsistent between the uncorrected and corrected images. A fixed correction mapping ensures coordinate consistency across multiple frames. For existing installations that cannot be calibrated, the correction step can be skipped, and the image can be directly annotated and cropped on the uncorrected image. In this case, simply using the same image space coordinate system in subsequent steps will maintain consistency with the requirements of the steps.

[0032] When cropping to obtain the target region of interest (PGI) sub-image, a filling mask is generated using the pixel coordinates of the polygon vertices. The area within the mask represents the PPI, while the area outside the mask is filled with constant gray values. Subsequently, the corresponding pixel block is copied into a new image data set based on the minimum bounding rectangle of the mask, serving as the PPI sub-image for that frame. This method of copying the filling mask maintains the accuracy of the polygon boundaries while providing a regular rectangular data block, facilitating subsequent row and column access in memory. To avoid jagged edges at the polygon boundaries, bilinear interpolation is used for the boundary pixels during cropping, and the interpolation result is rounded to an integer intensity value, thus maintaining consistency with the integer quantization of the image sensing device.

[0033] The row reading sequence number of each frame is read from the image sensor and used as a row-level time index. Specifically, the image sensor's driver interface is called to obtain the row number and row reading sequence information for each frame. When the driver interface also provides row time interval information, this information is recorded along with the row number, but the fixed exposure setting remains unchanged. When the driver interface only provides the row number, the row reading sequence number of each row is marked from top to bottom as 1 to the maximum row number of the image height. The reason for saving the row-level time index is that rolling exposure operates in a row-by-row exposure and row-by-row reading manner, with different rows in the same frame corresponding to different sampling times. In subsequent rolling shutter frequency demodulation, the brightness change sequence of the indicator lights within the same frame needs to be restored based on the row-level time index. The advantage of saving the row-level time index in this way is that it accurately locates the sampling time sequence corresponding to each row without making any changes to the image pixels, ensuring a clear timing reference for subsequent inferences from fringe boundaries to pulse structures.

[0034] To improve stability in strong light and backlight scenarios, a dynamic range pre-check can be performed before data acquisition. In the pre-check, select a reference area (32×32 pixels) around the indicator panel and another reference area around it. Check if the average brightness of the reference areas falls between 20% and 80% of the visible range under a fixed exposure. If the average brightness of the panel area is higher than 80%, it indicates that the panel is approaching saturation. At this point, rotate the lens polarizer to an angle perpendicular to the reflection direction until the average brightness of the panel area drops below 80%, and then reposition the polarizer. The reason for adding the polarizer is that under backlight or strong reflection, specular reflection on the panel surface significantly increases brightness and masks the actual light-emitting part of the indicator. Polarization selection can attenuate the specular reflection component and enhance the diffuse reflection component, improving the contrast between the indicator light area and the non-light-emitting area of ​​the panel. This results in clearer boundaries and more reliable color statistics in subsequent polygon annotation and cropping.

[0035] After completing the polygon annotation for the first frame, the perimeter length and area of ​​the polygon are immediately calculated and recorded for consistency checks. Before cropping each subsequent frame, the pixel coordinates of the polygon vertices are projected onto the current frame, and the same perimeter length and area are calculated. If the change compared to the first frame is less than or equal to 5%, the target region of interest sub-image is obtained directly by cropping according to the recorded pixel coordinates of the polygon vertices. If the change is between 5% and 10%, the operator is prompted to check for any minor displacement of the support and confirm whether to continue. If the change exceeds 10%, the current acquisition round is paused, and the annotation of the first frame is re-executed to ensure that the target region of interest sub-image is consistent with the actual panel position. Through this threshold verification based on area and perimeter, the relative offset between the camera and the panel can be quickly detected without introducing complex geometric calculations, thus ensuring spatial consistency in subsequent comparative analysis.

[0036] In some field applications, indicator panels contain multiple independent functional areas, such as closing, opening, fault, and power indicators. In such cases, an optional implementation using multiple polygon annotations can be adopted. Specifically, in the first frame, a separate polygon is annotated for each functional area, the vertex pixel coordinates of each polygon are recorded, and each polygon is assigned a unique name. In each frame, the corresponding target region of interest sub-image is cropped according to its respective polygon, and a row-level time index is saved for each. This is done because partitioning reduces non-target pixels within each sub-image, improving the accuracy of color statistics under strong light or backlight conditions, and providing a clear source correspondence for subsequent determination symbol generation.

[0037] In another alternative implementation, after the first frame is annotated, the pixel coordinates of the polygon vertices can be exported as a portable text file and imported into other devices using the same installation location information from the same type of image sensor. After importing, only minor adjustments to the vertex positions are needed on the first frame; typically, the movement of each vertex does not exceed 5 pixels to obtain a polygon that matches the field panel. This method facilitates rapid deployment of a batch of sites while maintaining consistency with the process of "cropping the target region of interest sub-image in each frame image based on the recorded pixel coordinates of the polygon vertices."

[0038] After acquiring at least 12 consecutive frames, annotating the first frame with polygons and recording the pixel coordinates of the polygon vertices, cropping each frame to obtain the target region of interest sub-image, and recording the row-level time index for each frame, these four types of data are archived frame by frame. Each frame entry contains the original color image, the target region of interest sub-image, the row-level time index, and a snapshot of the pixel coordinates of the polygon vertices used for annotation. The purpose of archiving is to ensure that subsequent processing can reproduce the preconditions of cropping and time indexing at any point in time, ensuring repeatability and stability in strong light and backlight scenes. Through the above process, step 1 completes the acquisition and target region of interest determination, and the obtained data directly meets the input requirements for subsequent steps such as rolling shutter frequency demodulation, quaternion color invariant generation, and decision symbol generation.

[0039] Step 2, Core Processing and Judgment Symbol Generation: Within the target interest region sub-image of each frame, perform rolling shutter frequency demodulation to generate physical demodulation labels, and perform quaternion color invariant generation to identify color valid indicator lights, and then perform a-contrario geometric test to generate geometric conclusions; align the physical demodulation labels, geometric conclusions and color valid indicator lights to generate judgment symbols.

[0040] In one specific implementation, rolling shutter frequency demodulation is first performed to generate physical demodulation labels. The goal is to directly extract the stripe bands generated by the frequency interaction between rolling exposure and indicator light emission rhythm from the line-level temporal differences, and to distinguish between flickering, constant brightness, constant darkness, and weak flickering based on the number of stripe bands and the movement relationship of the stripe phase starting point in adjacent frames. Specifically, the target area of ​​interest sub-image is converted into a brightness map, and a top-hat operation with a 3×3 structuring element is performed to obtain a highlight enhancement map. The reason for using the top-hat operation is that the indicator light emission area usually presents local bright patches, and the top-hat operation can increase the contrast between these patches and the surrounding background, making it easier for subsequent line differences to show positive and negative alternation. The adjacent line difference maps are calculated in the line direction, using the order of subtracting the second line from the first line, and then subtracting the third line from the second line, to obtain a difference map that only reflects the brightness change in the line direction. The reason for using adjacent rows for difference is that rolling exposure sampling results in a fixed, minute interval between the sampling times of adjacent rows within the same frame. The brightness changes of the indicator lights within this interval will appear as alternating signs in the difference image, forming clear bright-to-dark and dark-to-bright boundaries. Subsequently, the difference image is traversed column by column, starting from row number 1, checking the sign change of the difference values ​​pixel by pixel. Boundaries changing from positive to negative are recorded as bright-to-dark boundaries, and boundaries changing from negative to positive are recorded as dark-to-bright boundaries. These two types of boundaries are alternately paired to form bands. Bands with a height of less than 2 pixels are merged with adjacent bands of the same type to obtain a stable band set. The merging threshold is set to 2 pixels because bands below this height are often caused by random noise, and merging can reduce false stripes. The stable bands are then aggregated along adjacent columns to obtain stripe bands that run through the column direction. The number and spatial continuity of the stripe bands reflect whether the bright and dark stripes caused by rolling exposure are reliably captured. When the number of stripes is greater than or equal to 3, and the first dark-to-bright boundary appears from top to bottom in each column, the row number of this boundary is taken as the stripe phase start point of that column. In adjacent frames, observe whether the row number of the stripe phase start point moves monotonically downwards or monotonically upwards. Monotonous movement indicates that the stripe phase is steadily advancing, reflecting a stable heterofrequency effect between the indicator light's illumination rhythm and row-level reading; therefore, it is marked as flickering. When the number of stripes is equal to 0 and the proportion of bright pixels in the target area of ​​interest sub-image is greater than or equal to 90%, it is marked as constantly bright; when the number of stripes is equal to 0 and the proportion of bright pixels is less than or equal to 10%, it is marked as constantly dark; all other cases are marked as weak flickering. To further improve reliability under complex lighting conditions, a difference sequence can be calculated for the row numbers of the stripe phase start points according to the frame number within the acquired image sequence. When the absolute value of the difference sequence remains the same for 3 consecutive frames or increases only by a step size of 1 pixel, an equivalent heterofrequency envelope peak is considered to have appeared. Object understanding peak tuning is recorded on the frames where the equivalent heterofrequency envelope peak appears. The equivalent heterofrequency envelope peak is used to provide a stable physical modulation anchor point under different frequency combinations of light source and indicator light, so that the slow brightness fluctuations caused by ambient light no longer interfere with the flicker detection.In frames with physical demodulation peaks, the flicker conclusion is given priority; in frames without peaks, the judgment result of the aforementioned rules is retained as the physical demodulation label.

[0041] After completing the physical demodulation labeling, quaternion color invariant generation is performed to identify color valid indicator lights. The goal is to output a color invariant map unaffected by overall brightness changes under strong light, backlight, and partial saturation conditions, and to determine the primary color label within the color invariant map on a connected component basis, outputting color valid indicator lights. Specifically, the average values ​​of the red, green, and blue channels are calculated. The average of these three channel averages is used as a baseline value. Each channel is then gain-corrected by dividing the baseline value by the channel average value, resulting in corrected red, green, and blue channels. This channel gain correction eliminates the influence of illumination color shift on the overall level of the red, green, and blue channels, making color comparisons between different frames comparable. The arithmetic mean of the corrected red, green, and blue channels is used as the luminance, and quaternion components are formed in the order of luminance, red, green, and blue. Conjugate operations are performed on the quaternion components, and the square root of the sum of the squares of the four components is calculated as the joint intensity. Each of the four components is divided by the joint intensity to obtain a normalized quaternion component. The normalization process normalizes brightness variations, retaining only the red, green, and blue components that primarily reflect chromaticity information. The red, green, and blue components of the normalized quaternion component are used as the color invariant map, and a 3×3 median filter is applied to suppress isolated noise points. Saturated pixels are identified using the maximum output value of the image sensor, and a 3×3 closing operation is performed to expand the saturated region. The pixel values ​​in the saturated region are replaced with the average of adjacent unsaturated pixels before being included in subsequent statistics. Replacing the saturated region aims to prevent the shearing effect caused by highlights from pulling the proportions of the red, green, and blue channels in the wrong direction. Connected components are then labeled within the color invariant map. For each connected component, the pixel counts of the red, green, and blue components are counted, and the component with the highest count is used as the dominant color label for that connected component. If the difference in pixel counts between the red and green channels is less than or equal to 10% of the total pixel count within the connected component, and the pixel counts of both the red and green channels are greater than the pixel count of the blue channel, the dominant color label is set to yellow. Connected components with primary color labels are output as valid color indicator lights, and the center position, circumscribed rectangle, and pixel count of the connected components are recorded for subsequent geometric verification and alignment. The reason for using the above-mentioned processing chain based on quaternion components is that it can eliminate the overall brightness shift caused by strong light or backlight, retaining only the chromaticity information that is meaningful for judgment, thereby forming a stable distinction between the red, green, and yellow indicator lights of the real device.

[0042] After obtaining the spatial location and size information of the color-valid indicator lights, an a-contrario geometric test is performed to generate geometric conclusions. The purpose of this test is to independently verify that the geometric shape of the color-valid indicator lights indeed conforms to the circular or near-circular structure of the indicator lights, and simultaneously confirm the geometric consistency of the stripes being approximately straight lines, thus counteracting accidental connected regions caused by highly reflective or complex textured backgrounds. Specifically, a Sobel amplitude map is calculated for the target region of interest sub-image. After non-maximum suppression and double-threshold connection, an edge map is obtained. The proportion of edge pixels in the target region of interest sub-image is counted as the edge density baseline. The edge density baseline reflects the probability level of expected edge hits at any uniformly sampled point under random backgrounds. Subsequently, two sets of concentric sampling rings are generated at the center of the connected regions of the color-valid indicator lights, with radii of 1 and 2 times the indicator light diameter, respectively. The indicator light diameter can be directly taken as the shorter side length of the bounding rectangle of the connected region; the advantage of this method is its simplicity and proximity to the projected size of an actual circular LED. 36 points are sampled at equal intervals on each set of concentric sampling rings, and each point is checked to see if it hits an edge pixel on the edge map, resulting in two sets of hit counts. The hit count of each group is compared with the expected hit count of 36 points calculated based on the edge density baseline. When the hit count of both groups of concentric sampling rings is at least 8 more hits than the corresponding expected value, the connected component is confirmed as a geometrically valid indicator. The reason for setting at least 8 more hits is that the 36 equally spaced sampling points correspond to one orientation every 10 degrees. More than 8 hits means that at least about 80 degrees of arc length coincides with the real boundary, which has a very low probability of being triggered by accidental noise in a natural texture background. For the stripe structure, skeletonization is performed on the stripe stripe obtained by rolling shutter demodulation. Points are sampled every 5 pixels along the skeleton, and sliding windows of length 7 are constructed sequentially. The turning change count of adjacent sampling directions is calculated in each window. When the turning change count of any sliding window is less than or equal to 1, the stripe stripe is marked as a straight stripe. When the number of straight stripe stripes in a frame is greater than or equal to 3, the stripe structure of the frame is confirmed to be geometrically consistent. The reason for using straight-line stripes for judgment is that rolling exposures combined with uniform flickering will produce nearly parallel light and dark bands within the target area of ​​interest sub-image. If the background texture is strong but not caused by flickering, its skeleton direction will change frequently, making it difficult to meet the constraint of low turning change count. Based on the presence of geometrically valid indicator lights and whether the stripe structure is geometrically consistent, the output geometric conclusion is: geometrically passed, indicator light only passed, or stripe only passed. Geometrically passed indicates that the color and edge shape are consistent with the indicator lights and the rolling stripes conform to straight-line characteristics; indicator light only passed indicates that the color and geometric shape are valid, but the stripes have not yet satisfied the straight-line consistency; stripe only passed indicates that the stripes are straight-line consistent, but the color-valid indicator lights have not yet formed a stable connected region.

[0043] refer to Figure 3The schematic diagram of the a-contrario geometric inspection sampling ring described in this invention illustrates a complete method for verifying the geometric consistency of indicator lights and stripes. Figure 3 The left side of the diagram shows a sub-diagram illustrating the geometric verification principle of the indicator light. As shown, a circular indicator light exists within a gray background area. This indicator light has a clearly defined edge outline with its center point O as the center. The main body of the indicator light is filled with gray to represent its luminous area, and the edges are outlined with thick black lines to highlight its geometric shape. A solid dot and the text "Center O" are marked at the center of the indicator light as a reference point for geometric verification. Two sets of concentric sampling rings are drawn on the left side for geometric verification at the edge of the indicator light. The radius r1 of the first sampling ring (inner ring) is equal to one time the diameter d of the indicator light. This sampling ring is drawn with dashed lines, and 36 sampling points are evenly distributed on it, each represented by a small solid circle, with a sampling interval of 10 degrees. The radius r2 of the second sampling ring (outer ring) is equal to twice the diameter d of the indicator light. It is also drawn with dashed lines, and similarly, 36 sampling points are evenly distributed on it. The diagram illustrates the sampling parameters: inner ring radius r1 = 1d, outer ring radius r2 = 2d, where d is the indicator light diameter; the number of sampling points is 36; and the sampling interval is 10 degrees. The purpose of using a double-layer concentric sampling ring design is to verify whether the connected component possesses circular or near-circular geometric features by simultaneously sampling near the indicator light edge (inner ring) and in the area surrounding the indicator light (outer ring). If the edge hit counts of both the inner and outer rings are significantly higher than the random expected value, it proves that the connected component indeed corresponds to the actual indicator light structure, rather than a false positive connected component formed by background texture or random noise. Figure 3 The right side of the image shows the method for testing straight-line stripes. This section demonstrates two types of stripe skeleton examples. The upper example is a qualified stripe, whose skeleton is nearly straight. The points sampled every 5 pixels along the skeleton are arranged in a basically straight line, and the turning change of adjacent sampling directions is less than or equal to 1, indicating that the stripe conforms to the characteristics of parallel light and dark bands produced by rolling exposure. The lower example is a failed stripe, whose skeleton is zigzag, and the direction of the sampling points changes frequently, with a turning change greater than 1, indicating that the stripe may originate from complex background textures rather than indicator light flashing, and does not meet the criteria for straight-line stripes. The implementation details of the straight-line stripe test are as follows: First, the stripe band is skeletonized to obtain the center line of the stripe; then, a point is sampled every 5 pixels along the skeleton to form a point column; a sliding window of length 7 is constructed sequentially, and the turning change count of adjacent sampling directions is calculated within each window; when the turning change count of any sliding window is less than or equal to 1, the stripe band is marked as a straight-line stripe. The sampling interval is set to 5 pixels because this interval can maintain the sampling density while avoiding directional jitter caused by local pixel noise; the turning change threshold is set to 1 to ensure that the overall direction of the skeleton remains stable.

[0044] The generation of geometric conclusions follows the principle of independent verification. The geometric morphology verification of the color-valid indicator light and the linearity verification of the stripe are independent of each other, confirming the reliability of the detection results from both spatial morphology and temporal stripe dimensions. Through this dual verification mechanism, the present invention can effectively combat accidental connected regions and false stripes caused by high-reflectivity backgrounds, complex texture backgrounds, or changes in illumination, ensuring that the final generated judgment symbol has a high degree of confidence.

[0045] After completing the three parts, the physics demodulation label, geometric conclusion, and color valid indicator light are aligned to generate a decision symbol. Alignment is based on spatial overlap. The connected components of the color valid indicator light and the stripe are counted for overlapping pixels. When the overlapping pixel count accounts for more than 30% of the connected component pixel count, they are considered to be spatially matched. The reason for choosing 30% is that the indicator light's luminous area only needs to have sufficient area to overlap with the stripe to prove that the stripe comes from the indicator light and not the background. A value higher than 30% can mitigate the impact of irregular edges, slight viewing angle changes, and incomplete cropping on the degree of overlap. Under the premise of successful matching, a decision symbol for the frame is generated based on the combination of the physics demodulation label, the primary color label of the color valid indicator light, and the geometric conclusion. If the physics demodulation label is flashing and the primary color label is one of red, green, or yellow, and the geometric conclusion is either geometrically passed or only the indicator light passes, the output is red flashing, green flashing, or yellow flashing, respectively. If the physics demodulation label is constantly lit and the primary color label is one of red, green, or yellow, the output is red constantly lit, green constantly lit, or yellow constantly lit, respectively. When the physics demodulation label is normally dark and there is no valid color indicator light in the frame, the output is normally dark. When the decision symbol of two adjacent frames changes from solid green to flashing green or normally dark and then to solid red, the output switches from green to red; when the decision symbol of two adjacent frames changes from solid red to flashing red or normally dark and then to solid green, the output switches from red to green. The generated decision symbols are written into a sequence according to the frame number for subsequent parsing.

[0046] In one alternative implementation, the structuring element of the top-hat operation can be selected as 5×5 to accommodate indicator lights with larger diameters; when the resolution of the target interest region sub-image is higher than 1280×720, a 1×3 vertical mean filter is performed before row differencing to suppress random noise; during the connected component marking stage of the color invariant image, connected components with an area of ​​less than 50 pixels can be directly removed to reduce interference from small bright spots; the number of sampling points of the concentric sampling ring can be set to 48 or 72 to improve sensitivity to edge gaps, while the threshold of "at least 8 more hits" is proportionally adjusted to at least 10 more hits or at least 16 more hits; the sampling interval of the stripe skeleton can be set to 3 pixels to obtain more stable linear discrimination when the stripes are thinner due to long-distance installation. In another alternative implementation, the geometric check first performs a quick estimation of the center and diameter of the valid color indicator light. Then, it searches for a local maximum hit count by fine-tuning the radius near this center. The radius search range is 0.8 to 1.2 times the indicator light diameter, with a step size of 1 pixel, to accommodate differences in appearance size caused by different lens distortions and viewing angles. The advantage of using local maximum search is that it provides a radius that more closely fits the physical boundaries of the sampling loop without changing the overall process, thus ensuring a stable hit count exceeding the expected value even when strong light reflection causes edge blurring.

[0047] Step 3, WFST State Machine Parsing and Event Output: Construct a weighted finite state transition machine (WFST). Take the decision symbols generated in Step 2 as input and parse them using the WFST state machine to output event categories, where the event category is one of normal indication, operation prompt, or abnormal event. After parsing, overlay the target interest region outline, main color label, stripe band, and physical peak position on the corresponding frame to form an evidence map, and publish the event category via voice broadcast and platform push.

[0048] In one specific implementation paradigm, after step 2 generates decision symbols for each frame and provides the corresponding stripe and physical balancing peak positions, step 3 uses a weighted finite state transition machine (WFST) to parse the decision symbol sequence and outputs the event category and evidence map after parsing. First, prepare the input. Arrange the decision symbols obtained in step 2 in ascending order of frame number to form a decision symbol sequence. Each decision symbol carries the frame number of that frame, as well as the connected component position, stripe position, and physical balancing peak position of the color-valid indicator light within that frame, which will be used for evidence map overlay later. To ensure temporal consistency in strong light and backlight scenarios, it is recommended to parse at least 12 frames at a time; in scenarios with a fast on-site indicator rhythm, 20 frames can be covered to ensure that the switching behavior is fully included. Construct the WFST state machine. The state set includes initial, green constant state, red constant state, yellow constant state, green flashing state, red flashing state, yellow flashing state, switching state, alarm state, and end. The input alphabet is the set of judgment symbols generated in step 2, specifically including solid red, solid green, solid yellow, flashing red, flashing green, flashing yellow, green-to-red transition, and red-to-green transition. The output alphabet includes normal indicators, operation prompts, and abnormal events. The relationship between states and inputs is solidified into a transition table in tabular form. Each row gives the current state, input judgment symbol, next state, and output event category. For example, when reading flashing green in a solid green state, the next state is flashing green, and the output is an operation prompt; when reading flashes of the same color three times consecutively in any flashing state, the next state is an alarm state, and the output is an abnormal event. The advantage of using a tabular approach is that during parsing, only table entries need to be retrieved and explicit actions need to be executed, avoiding runtime ambiguity and facilitating consistent implementation across different platforms.

[0049] Frame-by-frame parsing is performed, with the parsing pointer initially pointing to the initial state. After reading the first determination symbol, the system enters the corresponding state according to the transition table and generates an output. For example, when the first determination symbol is a solid green, the state transitions from the initial state to the solid green state and outputs a normal indication. To support the conditional judgment of "continuous reading," an integer counter is established when entering any flashing state, initially set to 1. The counter is incremented by 1 when the next frame still shows flashing of the same color, and reset to 1 when a different color or solid green occurs. The reason for using an integer counter is that "continuous reading" requires a clear counting basis. The counter is updated synchronously with each state update, making it easy to implement in a real-time system and accurately triggering the condition of "three consecutive flashes of the same color." In the solid green or solid red state, to support the conditional judgment of "reading any solid green once," after reading a solid green, the system directly enters the corresponding solid green state according to the transition table and outputs a normal indication; in the switching state, after reading any solid green, the system directly enters the corresponding solid green state and outputs a normal indication. When reading a red state from a green state, the system directly enters the alarm state according to the transition table and outputs an abnormal event to cover any abnormal state transitions that occur without passing the switching decision. Parsing continues until the entire sequence is read or the end is reached.

[0050] Once the event is formed, the output generated during the parsing process is written into the event list in chronological order. Each event includes the event category, the first trigger frame number, the current state, the corresponding judgment symbol, and a readable description. For example, if three consecutive flashes of the same color occur in any flashing state, triggering an alarm state, the description is constructed as "Three consecutive flashes of the same color, processed as an alarm state," and the first trigger frame number is recorded as the frame number of the first occurrence of that color flash. This construction method allows maintenance personnel to immediately grasp the triggering basis when viewing events. Multiple events may be generated in a single parsing process; the event list is arranged in ascending order of frame number, facilitating the comparison with evidence. Figure 1 One-to-one correspondence.

[0051] Generate an evidence map. For each event, copy a target interest region sub-map in the corresponding frame as the base map, and overlay the target interest region outline, main color label, stripe band, and physics adjustment peak position. The target interest region outline is drawn using the pixel coordinates of the polygon vertices recorded in step 1. The vertex connection line width is 2 pixels, and the color is fixed to white to ensure visibility in both strong light and backlight scenes. The main color label is read from the main color label of the connected component of the color valid indicator obtained in step 2. The label text is placed above the bounding rectangle of the connected component, with a text height of 20 pixels and the text color consistent with the main color label. The stripe band is displayed as a semi-transparent horizontal stripe, with the stripe height consistent with the actual height of the stripe band and an opacity of 30%. This shows the appearance position of the stripe without obscuring the details of the base map. The physics adjustment peak position is marked with a short vertical line at the top edge of the frame, with the word "peak" below the short line. The short line length is 15 pixels. Presenting the four types of overlay elements simultaneously allows for a direct correspondence between the event and the image evidence, facilitating visual verification under complex lighting conditions.

[0052] After the analysis is complete, the event list will be matched with the corresponding evidence. Figure 1 The platform recommends using a combination of image files and structured text for push notifications. The structured text includes the event category, frame number, status name, and judgment symbol, where the event category is limited to one of three: normal indication, operation prompt, or abnormal event. The voice broadcast generates a natural language description for each event, such as "Frame 15, red flashing occurred 3 times consecutively, judged as an abnormal event, please check the circuit." The voice sampling rate can be 16000 Hz, and the bit rate can be 64 kilobits per second. The broadcast is triggered immediately after the event occurs. This parallel image and text publishing approach can simultaneously satisfy two usage scenarios: rapid perception and detailed review. Upon receiving the voice notification, operators can immediately focus on key points and then confirm the situation by comparing the stripe pattern with physical indicators to understand the peak location.

[0053] In one optional implementation, to reduce frequent broadcasts caused by short-term jitter, a one-time time suppression can be introduced before event output. When the same event category appears repeatedly in adjacent frames and the frame number difference between the two events is less than or equal to 2, only the first occurrence of the event is retained, and the physical tuning peak position of the subsequent event is superimposed on the evidence map as a reference marker, thereby preserving key information while reducing repetitive prompts. In another optional implementation, the overlay color of the evidence map can be adaptively selected based on the average brightness of the base map. When the average brightness of the target area of ​​interest sub-map is higher than 70%, the overlay color is black to enhance contrast; when the average brightness is lower than 30%, the overlay color is white. When the average brightness is between 30% and 70%, the overlay color is the same as the main color label, thus maintaining good visibility in both bright and backlit scenes.

[0054] In another alternative implementation, the readability of the switching determination can be enhanced by adding annotations. When a green-to-red or red-to-green switch is detected, an arrow segment with a length approximately equal to the center distance is drawn on the evidence map connecting the center of the connected domains of the valid color indicator lights in the two frames. The arrow points to the position of the valid color indicator light after the switch, and the word "Switch" is labeled next to the arrow. This annotation can intuitively show the time and spatial location of the switch, and is particularly suitable for panels with multiple indicator lights closely arranged, helping to quickly locate the target.

[0055] Example 2: In this example, the image sensing device is set to a resolution of Pixels, frame rate 30 frames per second, fixed focal length and fixed exposure. 12 consecutive frames of color images are acquired, and the frame number is recorded as follows. The target region of interest polygon was manually annotated in frame 1, with vertex pixel coordinates as follows: Together with two rounded corner points, a total of 8 vertices are formed. The minimum bounding rectangle size of the target interest region subgraph obtained by cropping is approximately... Pixels. Row-level time indexing is provided by the image sensor, with row numbers from top to bottom as follows: .

[0056] Within each frame's target interest region sub-image, sequentially perform rolling shutter frequency demodulation, a-contrario geometric verification, and quaternion color invariant generation, and align them to obtain the judgment symbol.

[0057] I. Rolling shutter frequency demodulation (in frames) (The calculation process is given as an example): Convert the target area of ​​interest sub-image into a brightness map, denoted as... .in Indicates that in frame number The frame and line coordinates are Column coordinates are The brightness value ranges from 0 to 255. Performing top-hat operations yields the highlight enhancement image. Defined as ;in Represents the morphological opening operation. express The square structural element. Then, the difference graph between adjacent rows is calculated along the row direction. ;in For action With Action The brightness difference can be positive or negative. For each column Top-down inspection The sign change is recorded as follows: a change from positive to negative is designated as a "bright-to-dark boundary," and a change from negative to positive is designated as a "dark-to-bright boundary." Adjacent "bright-to-dark boundaries" are paired with "dark-to-bright boundaries" to form bands. Bands with a height of less than 2 pixels are merged with adjacent bands of the same type to form a stable band set. Connectivity aggregation is performed on the stable bands in the column direction to obtain the number of stripe bands. .

[0058] In this frame In the calculation, the statistical result is In each column The first "dark-to-light boundary" line appearing from top to bottom is denoted as: To compare the fringe phase of adjacent frames, 50 column coordinate samples with equal spacing were selected within the region. For each sample column, calculate the row coordinate difference of the fringe phase start point between adjacent frames. ;in Indicates from frame to frame List The change in the row coordinates of the fringe phase start point. For frames Representative list Observed From this, we can conclude that... Statistical analysis across 50 columns shows that the vast majority of samples... The absolute value is 4 in both adjacent comparisons. Based on the construction rule of "equivalent heterofrequency envelope peaks," detection is performed when the absolute value of the difference sequence remains the same or increases by only one pixel step within three consecutive frames. The above records help to understand peak adjustment. Combined with... and The fact that the frame number changes monotonically with the frame number, The physical modulation label for all frames is "flickering". For comparison, the frame... The statistics are as follows And the proportion of bright pixels was measured. ;in Indicates in frame The target focus is on the proportion of bright pixels in the sub-image of the region of interest, with the numerator being the pixel set count. The middle obtained respectively According to the rule of "a percentage greater than or equal to 0.90", it is judged as "always on". Measured again and It was also judged to be "Chang Liang".

[0059] II. Generation of Quaternion Color Invariants (in frames) (The calculation process is given as an example) In the frame Within the target area of ​​interest sub-image, denoted as color pixels. ,in These represent the pixel values ​​for the red, green, and blue channels, respectively, ranging from 0 to 255. First, calculate the average value of the three channels within the region. ;in Representing frames respectively The average values ​​of the red, green, and blue channels. This represents the total number of pixels. Let... ;in This represents the average of the three channels. Gain correction is applied to each pixel channel. ;in These are the three-channel pixel values ​​after gain correction. Brightness is defined. ;in Represents the brightness pixel value. Constitutes a quaternion component. ;in The four components are brightness, red, green, and blue, respectively. Conjugate is defined. ;in Represents the conjugate of quaternion components. Defines joint strength. ;in This represents the modulus of the quaternion component. It is then normalized. ;in For normalized quaternion components, These represent the unitized brightness and three channels, respectively. The color invariant plot is taken as follows. ;in The three components are used for subsequent color statistics. Do Median filtering. Identifying sets of saturated pixels. ;in This represents the set of saturated pixels. Do The closing operation is expanded and replaced with the mean of the unsaturated pixels in the neighborhood. Then... Mark the connected components on the top, and for each connected component... Statistical Three-Component Count ; ;in These represent the pixel counts that were identified as the red, green, and blue principal components within the connected component, respectively. The corresponding component determines the primary color label. If the following conditions are met... ;and Then the primary color tag is set to yellow. In this frame... main connected components The above statistics show that The primary color tag is green. Output as a color-valid indicator light, and simultaneously record its center coordinates. (Measured in subgraph coordinates) Length of the shorter side of the circumscribed rectangle Pixels and Pixel Count .

[0060] III. a-contrario geometrical test (still based on frames) For example): Calculate the Sobel magnitude and perform non-maximum suppression and double thresholding to obtain the edge map. Statistical edge density baseline ;in Indicates the proportion of edge pixels. Measured in this frame. .by Center and diameter estimation determines the radius of the two sets of concentric sampling rings ;in These represent the radii of the inner and outer rings, respectively. 36 angle samples are taken at equal intervals on each sampling ring. And map to image coordinates, and count the hits. Expected hit count is Compare the differences Since both groups meet the condition of "at least 8 more hits", it is confirmed. This is a geometrically valid indicator. The stripe bands obtained from the rolling shutter frequency demodulation are skeletonized and sampled every 5 pixels along the skeleton. A sliding window of length 7 is used to count the turning changes, and the maximum value of the turning change count within all windows is 1. The number of stripe bands is 4, which satisfies the condition that "the number of straight stripes is greater than or equal to 3". Therefore, the stripe structure of this frame is geometrically consistent, and the geometric conclusion is that it passes geometrically.

[0061] IV. Alignment and Decision Symbol Generation (Frame) ): Calculate the connected components of the valid color indicator lights The overlapping area is obtained by counting the overlapping pixels with the stripe band. ;in This represents the number of overlapping pixels. The overlap ratio is obtained by normalizing the pixel count of the connected components. ;because Match successful. The physics tone label is "flickering," the primary color label is "green," and the geometric conclusion is "geometric pass." According to the combination rule of "flickering and geometric pass," the frame... The determination symbol is green flashing. Processing 12 frames in the same way, the frame-by-frame determination symbols obtained in this example are: Step 3: Use the WFST state machine to parse the above decision symbol sequence and generate event categories and evidence diagrams.

[0062] Set the input alphabet as the set of decision symbols, and the output alphabet as normal indicators, operation prompts, and abnormal events. Set the state set as initial, solid green, solid red, solid yellow, flashing green, flashing red, flashing yellow, toggle state, alarm state, and end. Parsing starts from the initial state and reads... The green light remains constantly lit, transitioning to a constant green state and outputting a normal indicator. Read... Evergreen and The green light remains constantly lit, maintaining this state and outputting a normal indicator. (Read) The screen flashes green, transitions to green flashing state, outputs an operation prompt, and establishes a same-color flashing counter. juxtaposition Read The green flashing, maintain the green flashing state and place Output operation prompts. Read The green flashes, placed An alarm state is triggered based on "reading three consecutive flashes of the same color in any flashing state", and an abnormal event is output. (Read) The green light remains constantly lit, transitioning from the alarm state to the constantly lit green state and outputting the exception event. Read... If the green light remains constant, the process ends and a normal output indicator is displayed. The remaining input is then processed as finished, and sequence parsing is complete.

[0063] The timeline of events obtained from this analysis is as follows: Output normal indication; in Output operation prompts; in Output exception events; in Output exception events; in Output is normal. (In frame) Overlay the following graphic elements onto the target area of ​​interest submap: Draw the outline of the target area of ​​interest using the polygon vertices saved in step 1 (line width 2 pixels, color white); Label the main color "green" above the outer rectangle of the color valid indicator (font height 20 pixels, text color green); Display the stripe band with a semi-transparent overlay (transparency...). (The height of the band matches the actual height of the stripe band). The peak position is identified by a short, vertical line marker along the top edge of the image, and the character "peak" (15 pixels long) is labeled below it. Simultaneously, the text "Frame number 6, judgment symbol green flashing, status alarm state, event category abnormal event" is written in the lower right corner of the image. A natural language broadcast is generated: "Frame 6, green flashing occurred 3 times consecutively, judged as an abnormal event, please check the circuit." The audio sampling rate is 16000 Hz, and the bit rate is 64 kilobits per second. The platform pushes an image file containing structured text (event category, frame number, status name, judgment symbol) and evidence images.

[0064] refer to Figure 2The figure shows an experimental curve of stripe phase shift detection in an embodiment of the present invention. Figure 2 This includes a two-dimensional coordinate system, where the horizontal axis represents the frame number and the vertical axis represents the stripe phase start row number (unit: pixels). The horizontal axis scale ranges from 0 to 20 frames, marked in 2-pixel intervals; the vertical axis scale ranges from 0 to 500 pixels, marked in 100-pixel intervals. A grid is drawn within the coordinate system to facilitate data reading. In this embodiment, Figure 2 Three different experimental curves are shown, corresponding to the changes in the stripe phase start row number as a function of frame number at flash frequencies of 50Hz, 25Hz, and 100Hz. The first curve, plotted as a solid line, represents the stripe phase movement trajectory at a flash frequency of 50Hz. Starting from coordinate point (0, 400), this curve exhibits an approximately linear downward trend, ending near coordinate point (20, 20). Within a 20-frame observation period, the stripe phase start row number gradually moves from row 400 to row 20, demonstrating a stable downward movement. The slope of this curve is approximately -19 pixels / frame, reflecting the stable heterofrequency interaction between the 50Hz flash frequency and the image sensor's rolling exposure frequency. The second curve, plotted as a long dashed line (dashed line segment length 10, interval 5), represents the stripe phase movement trajectory at a flash frequency of 25Hz. The curve exhibits a periodic jump characteristic, rapidly decreasing from coordinate point (0, 400) to near coordinate point (10, 40) within the first 10 frames. Then, a phase loop occurs between frames 10 and 11, jumping to line 480 before continuing downwards. This periodic jump phenomenon indicates that at a flicker frequency of 25Hz, the fringe phase completes a full cycle within the effective line range of the image sensor, demonstrating the rapid phase advancement characteristics of lower-frequency flicker. The third curve, plotted as a short dashed line (segment length 5, interval 5), represents the fringe phase movement trajectory at a flicker frequency of 100Hz. Starting from coordinate point (0, 400), this curve shows a slow, linear downward trend, moving only from line 400 to near line 210 within a 20-frame observation period. The slope of this curve is approximately -9.5 pixels / frame, significantly less than the slope under 50Hz conditions, indicating that the fringe phase advancement speed is relatively slow at higher flicker frequencies. Figure 2 A legend box is provided on the right side, clearly indicating the line types corresponding to the three flicker frequencies: solid line for 50Hz, long dashed line for 25Hz, and short dashed line for 100Hz. The legend box has a black border and a white background to ensure readability in black and white printing. From Figure 2The experimental results show that this invention can effectively identify the blinking states of indicator lights at different frequencies by monitoring the movement pattern of the stripe phase start line number in a continuous frame sequence. When the stripe phase start line exhibits a stable movement that is monotonically decreasing or monotonically increasing, it indicates the presence of a stable heterofrequency interaction, which can be used to determine that the indicator light is blinking. The rate of phase movement is directly related to the frequency difference between the indicator light blinking frequency and the image sensor line reading frequency: the larger the frequency difference, the faster the phase movement; the smaller the frequency difference, the slower the phase movement.

[0065] While specific embodiments of the present invention have been described above, those skilled in the art should understand that these specific embodiments are merely illustrative. Those skilled in the art can omit, substitute, and modify the details of the above methods and systems in various ways without departing from the principles and essence of the present invention. For example, combining the above method steps to perform substantially the same function and achieve substantially the same result according to substantially the same method falls within the scope of the present invention. Therefore, the scope of the present invention is defined only by the appended claims.

Claims

1. A method for detecting and analyzing inspection defects of a traction substation, characterized in that, The method comprises the following steps: Step 1, collecting and determining the target area of interest: deploying an image sensing device with rolling exposure at a fixed point of the traction substation, setting a fixed focal length and a fixed exposure, and continuously collecting not less than 12 frames of color images; in the first frame, labeling the target area of interest of the indicator panel in a polygonal manner, recording the pixel coordinates of the polygon vertices, and cutting the target area of interest subgraph in each frame according to the recorded pixel coordinates of the polygon vertices; Reading the row read order number of each frame from the image sensing device as a row-level time index; Step 2, core processing and symbol generation: in each frame of the target area of interest subgraph, performing rolling shutter heterodyne demodulation to generate a physical demodulation label, and performing quaternion color invariant generation to identify a color-effective indicator light, and then performing a-contrario geometric verification to generate a geometric conclusion; Aligning the physical demodulation label, the geometric conclusion and the color-effective indicator light to generate a decision symbol; Step 3, WFST state machine analysis and event output: constructing a weighted finite state transition machine (WFST) state machine, taking the decision symbol generated in step 2 as input, and analyzing the input by the WFST state machine to output an event category, wherein the event category is one of a normal indicator, an operation prompt or an abnormal event; and after the analysis is completed, superimposing the target area of interest contour, the main color label, the stripe band and the physical demodulation peak position on the corresponding frame to form an evidence graph, and publishing the event category in the form of voice broadcast and platform push; The rolling shutter heterodyne demodulation comprises: converting the target area of interest subgraph into a luminance graph, and performing top-hat operation with a structure element of 3*3 to obtain a highlight enhancement graph; calculating a difference graph of adjacent rows in the row direction for the highlight enhancement graph, wherein the difference is obtained by subtracting the (n+1)th row from the nth row, and the difference graph only reflects the brightness change in the row direction; traversing the difference graph in the column direction, recording the bright-to-dark boundary from positive to negative and the dark-to-bright boundary from negative to positive, and alternately pairing the two types of boundaries to form a band region; merging the band regions with a height of less than 2 pixels and adjacent band regions of the same type to obtain a stable band region set; positioning the row number of the first dark-to-bright boundary appearing from top to bottom in each column as the stripe phase starting point of the column; according to the number of stripe bands and the row number offset of the stripe phase starting point in adjacent frames, the physical demodulation label is generated according to the following rules: when the number of stripe bands is greater than or equal to 3 and the stripe phase starting point moves monotonously downward or monotonously upward in adjacent frames, it is recorded as flicker; when the number of stripe bands is equal to 0 and the proportion of highlight pixels in the target area of interest subgraph is greater than or equal to 90%, it is recorded as constant bright; when the number of stripe bands is equal to 0 and the proportion of highlight pixels is less than or equal to 10%, it is recorded as constant dark; and the rest is recorded as weak flicker. The a-contrario geometric verification includes: calculating a Sobel amplitude graph for the target attention region subgraph, and obtaining an edge graph through non-maximum suppression and double threshold connection, and counting the proportion of edge pixels in the target attention region subgraph as an edge density baseline; at the center of the connected domain of the color effective indicator light generated in step 2, two sets of concentric sampling rings with a radius of 1 times and 2 times the diameter of the indicator light are generated; 36 points are sampled at equal intervals on each set of concentric sampling rings, and the hit count falling on the edge graph is counted; the hit count is compared with the expected value of the edge density baseline x 36, and when the hit counts of the two sets of concentric sampling rings are all more than at least 8 hits than the corresponding expected value, it is confirmed that the corresponding connected domain is a geometrically effective indicator light; performing skeletonization on the fringe strip obtained by the rolling shutter heterodyne demodulation in step 2, sampling a point column every 5 pixels along the skeleton, and constructing a sliding window with a length of 7 in order, calculating the sign sequence of the direction angle of the 5 adjacent directions in each sliding window, and counting the turning change count, when the turning change count of any sliding window is less than or equal to 1, the fringe strip is marked as a straight line type fringe; when the number of straight line type fringes in a frame is greater than or equal to 3, it is confirmed that the fringe structure is geometrically consistent; according to whether the geometrically effective indicator light exists and whether the fringe structure is geometrically consistent, a geometric conclusion is output, wherein: when the geometrically effective indicator light exists and the fringe structure is geometrically consistent, a geometric pass conclusion is output; when the geometrically effective indicator light exists but the fringe structure geometrically consistent condition is not triggered, only the indicator light pass conclusion is output; when the fringe structure geometrically consistent condition is triggered but the geometrically effective indicator light has not appeared, only the fringe pass conclusion is output; The generation of the judgment symbol includes: in the same frame, the object demodulation label, the geometric conclusion and the color effective indicator light are matched in pixel position, and the overlapping pixel count of the connected domain of the color effective indicator light and the fringe strip is used as the criterion, and when the overlapping pixel count accounts for more than or equal to 30% of the proportion of the connected domain pixel count, it is determined that the matching is successful; when the matching is successful, according to the combination of the object demodulation label, the main color label of the color effective indicator light and the geometric conclusion, the judgment symbol is generated.

2. The method of claim 1, wherein, The generation of the quaternion color invariant includes: calculating channel average values of red, green and blue channels, calculating an average of the three channel average values as a reference value, performing gain correction on each channel according to a proportion of the channel average value divided by the reference value, and obtaining corrected red, green and blue channels; taking an arithmetic average of the corrected red, green and blue channels as luminance, and arranging the luminance, red, green and blue in order to form a quaternion component; performing a conjugate operation on the quaternion component, calculating a square root of a square sum of the four components as joint intensity, and dividing the four components by the joint intensity to obtain a unitized quaternion component; taking red, green and blue components of the unitized quaternion component as a color invariant image; identifying saturated pixels with a maximum output value of the image sensing device, performing a 3*3 close operation to expand the saturated area, and replacing the saturated area with a mean value of adjacent non-saturated pixels; performing connected domain labeling in the color invariant image, and determining a dominant color label according to pixel counts of red, green and blue components in each connected domain, and taking the connected domain with the dominant color label as a color effective indicator light output.

3. The method of claim 2, wherein, The heterodyne demodulation of the rolling shutter further includes: calculating a difference sequence of the row number of the fringe phase starting point according to the frame number in the collected image sequence, constructing an equivalent heterodyne envelope peak when the absolute value of the difference sequence remains the same or increases by only 1 pixel in the next 3 frames, and recording the physical demodulation peak on the frame where the equivalent heterodyne envelope peak appears.

4. The method of claim 3, wherein, The determination of the dominant color label includes: when the pixel count difference between the red channel and the green channel is less than or equal to 10% of the total pixel count in the connected domain, and the pixel counts of the red channel and the green channel are both greater than the pixel count of the blue channel, setting the dominant color label as yellow.

5. The method of claim 4, wherein, The generation of the determination symbol further includes: when the matching is successful, the physical demodulation label is flickering, the dominant color label is one of red, green or yellow, and the geometric conclusion is geometric passing or only indicator light passing, outputting a red flickering, green flickering or yellow flickering determination symbol; when the matching is successful, the physical demodulation label is constant light, and the dominant color label is red, green or yellow, outputting a red constant light, green constant light or yellow constant light determination symbol; when the physical demodulation label is constant dark and the color effective indicator light does not exist, outputting a constant dark determination symbol; when the determination symbols of the adjacent two frames change from green constant light to green flickering or constant dark and then to red constant light, outputting a green-to-red switching determination symbol; when the determination symbols of the adjacent two frames change from red constant light to red flickering or constant dark and then to green constant light, outputting a red-to-green switching determination symbol.

6. The method of claim 1, wherein, The input alphabet of the WFST state machine is the set of determination symbols generated in step 2, the output alphabet is the set of normal indicators, operation prompts and abnormal events, and the state set includes: initial, green constant light state, red constant light state, yellow constant light state, green flickering state, red flickering state, yellow flickering state, switching state, alarm state and end.

7. The method of claim 6, wherein, The WFST state machine analysis includes the following state transition rules: reading a green flickering determination symbol in the green constant light state, entering the green flickering state and outputting an operation prompt; reading a red constant light determination symbol in the green constant light state, entering the alarm state and outputting an abnormal event; reading a red-to-green switching determination symbol in the red constant light state, entering the switching state and outputting an operation prompt; In any flickering state, the same color flickering determination symbol is read for 3 times continuously, the alarm state is entered and an abnormal event is outputted; In any flickering state, the same color constant determination symbol is read for 2 times continuously, the corresponding constant state is returned and a normal indication is outputted; in the alarm state, any constant determination symbol is read, the corresponding constant state is entered and an abnormal event is outputted, and then any constant determination symbol is read again, the end is entered and a normal indication is outputted.

Images