A method and system for real-time interpretation of dense LED flicker frequency with integrated jitter compensation and confidence assessment

Abstract

The application discloses a dense LED flicker frequency real-time interpretation method and system integrating jitter compensation and confidence evaluation, which is applied to the technical field of data processing.The application firstly performs gray-scale, contrast enhancement and denoising pretreatment on a continuous video stream, and then detects dense LEDs based on an improved YOLOv8 architecture, and judges the bright and dark states through the mean value of the V component of HSV and a dynamic threshold value.Subsequently, the coordinate is normalized and discretized, combined with DBSCAN clustering and cross-frame time sequence correlation to generate an LED time sequence track, linear interpolation is performed on the missing data, and binarization is performed, the flicker frequency is calculated through zero-crossing detection and sampling frame rate.Meanwhile, a double-factor confidence model is constructed to quantify reliability, and the optical flow method with adaptive computing power and coordinate correction are adopted to realize jitter compensation, and finally, the LED flicker frequency, state and confidence result are output.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for real-time interpretation of dense LED flicker frequency that integrates jitter compensation and confidence assessment. Background Technology

[0002] Currently, the maintenance of equipment in mechanical rooms in industries such as railways, power, and oil and gas mainly relies on regular manual inspections. This type of equipment (such as the fully electronic interlocking train control cabinet in railway electrical control rooms) is typically equipped with a large number of densely arranged LED indicator lights. Their status (always on, always off, flashing at a specific frequency) directly reflects the equipment's operating condition and is a key aspect of inspections.

[0003] The inherent drawbacks of manual inspection are as follows: low inspection efficiency, difficulty in achieving real-time monitoring, and significant challenges in special scenarios such as unmanned stations in high-altitude areas; for LED indicator lights with high flashing frequencies, it is impossible to accurately determine whether their flashing frequency and status are normal by visual observation alone; the dense arrangement and large number of LED indicator lights can easily lead to visual fatigue of staff during long-term inspections, causing potential hazards to be overlooked, and making it difficult to meet the strict requirements of industry inspection standards.

[0004] Existing intelligent inspection technologies have the following limitations: Current research on intelligent inspection robots is mostly limited to image acquisition and transmission, relying on manual subsequent recognition, and lacks a refined recognition module for dense LED indicator lights, especially in core functions such as lamp position logic recognition and frequency tracking. Existing LED recognition technologies are mostly applicable to simple scenarios where the LEDs are large, sparsely arranged, and only need to determine their on / off state. For complex scenarios where the LEDs are densely arranged, small in size, numerous, have high flashing frequency, large differences in brightness, and require independent frequency tracking, there is a lack of effective solutions. Summary of the Invention

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0006] A real-time interpretation method for dense LED flicker frequency integrating jitter compensation and confidence assessment includes: acquiring continuous video stream data, performing grayscale conversion, contrast enhancement, and noise reduction preprocessing to obtain an enhanced image; based on an improved YOLOv8 lightweight architecture, inserting a CBAM attention module and adding a small target detection layer to detect dense LED targets and outputting bounding box coordinates and confidence scores, calculating the mean V component of the HSV value of each detection box, and determining the LED on / off state through a dynamic threshold; converting the absolute coordinates of the LEDs to normalized relative coordinates, then discretizing them into adaptive grid relative coordinates, using DBSCAN density clustering for intra-frame spatial clustering, constructing a spatiotemporal correlation matrix, and combining positional similarity and state weights for cross-frame temporal correlation clustering. The system generates time-series trajectories for the same physical LED; processes the LED on / off state sequences corresponding to the time-series trajectories, uses linear interpolation to supplement data and binarizes frames that occasionally fail to be detected, extracts the rising edge of the state sequence using zero-crossing detection, and calculates the LED flicker frequency by combining the video sampling frame rate; constructs a two-factor confidence evaluation model based on the mean and standard deviation of the detection coverage and flicker interval to quantify tracking reliability, determine the LED's always-on / always-off state and tracking stability; estimates the global motion vector using the Lucas-Kanade sparse optical flow method or the Farneback dense optical flow method, performs cumulative jitter compensation through dynamic ROI adjustment and coordinate drift correction models, and outputs the dense LED flicker frequency, state determination results, and confidence evaluation results.

[0007] Another aspect of this application discloses a real-time interpretation system for dense LED flicker frequency integrating jitter compensation and confidence assessment. The system includes: a video stream preprocessing module for acquiring continuous video stream data, optimizing the original video frames through grayscale processing, contrast enhancement technology, and denoising algorithms to generate an enhanced image dataset adapted for subsequent detection, providing high-quality input for LED target recognition; a dense LED detection and state determination module for accurately detecting dense LED targets and outputting bounding box coordinates and confidence scores based on an improved YOLOv8 lightweight architecture, inserting a CBAM attention module, and adding a small target detection layer, calculating the mean V component of the HSV value within the detection box, and outputting the LED on / off state based on dynamic threshold determination rules; and a coordinate transformation and temporal clustering module for converting the absolute coordinates of the LEDs into normalized relative coordinates, then discretizing them into adaptive grid relative coordinates, completing intra-frame spatial clustering through DBSCAN density clustering, and constructing a spatiotemporal correlation matrix. The system combines positional similarity and state weights to achieve cross-frame temporal correlation clustering, generating temporal trajectories for the same physical LED. A flicker frequency calculation module processes the LED on / off state sequences, using linear interpolation to complete data for frames with occasional detection failures and performing binarization. It extracts the rising edge of the state sequence using zero-crossing detection and calculates the LED flicker frequency based on the video sampling frame rate. A confidence assessment and state determination module constructs a two-factor confidence assessment model based on the mean and standard deviation of detection coverage and flicker intervals, quantifying tracking reliability and determining the LED's constantly on / off state and tracking stability based on frequency thresholds. A jitter compensation and result output module selects either the Lucas-Kanade sparse optical flow method or the Farneback dense optical flow method to estimate the global motion vector based on hardware computing power. It achieves cumulative jitter compensation through dynamic ROI adjustment and coordinate drift correction models, ultimately outputting the dense LED flicker frequency, state determination results, and confidence assessment results.

[0008] According to another aspect of this application, an electronic device includes: a first processor; and a memory for storing executable instructions of the first processor; wherein the first processor is configured to execute the above-described method for real-time interpretation of dense LED flicker frequency integrating jitter compensation and confidence assessment by executing the executable instructions.

[0009] According to another aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a second processor, implements the above-described method for real-time interpretation of dense LED flicker frequency with integrated jitter compensation and confidence assessment.

[0010] Its beneficial effects are as follows: This invention provides a method and system for real-time interpretation of the flicker frequency of dense LEDs that integrates jitter compensation and confidence assessment. First, the video stream is preprocessed, and dense LEDs are detected by improving the YOLOv8 architecture (inserting a CBAM module and a small target detection layer). The on / off state is determined based on the HSV dynamic threshold. The LED timing trajectory is generated by normalized-grid coordinate transformation and DBSCAN clustering and cross-frame temporal correlation. After interpolation to complete the missing data, the flicker frequency is calculated by zero-crossing detection. The reliability is quantified by combining a two-factor confidence model, and jitter compensation is achieved by using an optical flow method and coordinate correction adapted to the computing power. Finally, accurate results are output.

[0011] This invention aims to solve the challenge of stable tracking of densely labeled LEDs. It combines coordinate transformation with spatiotemporal clustering to adapt to dense, small target scenarios; it has strong anti-jitter capabilities, with mesh discretization and optical flow compensation working together to suppress coordinate drift caused by camera vibration; it provides accurate frequency calculation, with interpolation completion and zero-crossing detection ensuring the integrity of time-series data and meeting real-time interpretation requirements; it is lightweight and adaptable, with algorithm optimization and dynamic ROI adjustment reducing computational overhead and adapting to inspection robot deployment; and it offers high reliability, with a two-factor confidence assessment quantifying tracking stability, providing a reliable decision-making basis for industrial inspection. Attached Figure Description

[0012] Figure 1 A flowchart of a real-time interpretation method for dense LED flicker frequency that integrates jitter compensation and confidence assessment is provided in an embodiment of the present invention;

[0013] Figure 2 This is a schematic diagram of a module for a real-time interpretation system of dense LED flicker frequency that integrates jitter compensation and confidence assessment, provided as an embodiment of the present invention. Detailed Implementation

[0014] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention. Figure 1 This application describes a method and system for real-time interpretation of dense LED flicker frequency that integrates jitter compensation and confidence assessment, according to exemplary embodiments thereof.

[0015] In this application embodiment, a method and system for real-time interpretation of dense LED flicker frequency integrating jitter compensation and confidence assessment is provided, such as... Figure 1 As shown:

[0016] S101: Acquire continuous video stream data, perform grayscale conversion, contrast enhancement, and noise reduction preprocessing to obtain an enhanced image.

[0017] In one implementation, the input data is a continuous video stream. Each frame is used as a basic processing unit, and the t-th frame is denoted as It. The video stream needs to cover a scene of dense LED indicator arrays in machinery rooms of industries such as railways and power, ensuring that the acquired images clearly show the arrangement and brightness changes of the LED indicators, meeting the basic data requirements for subsequent detection and analysis. Each acquired color image It undergoes grayscale conversion. By extracting the brightness information of each pixel in the image, the three-dimensional RGB color image is converted into a one-dimensional grayscale image. This processing simplifies subsequent calculations, reduces data redundancy, and preserves the brightness difference characteristics between the LED indicators and the background, laying the foundation for subsequent contrast enhancement and noise reduction processing.

[0018] For the grayscale image, contrast enhancement technology is used to improve the brightness difference between the LED indicator and the surrounding background. Since different LED indicators vary significantly in brightness due to manufacturing processes, signal strength, and other factors, contrast enhancement strengthens the brightness characteristics of bright LEDs and the distinguishability of dark LEDs, enabling more accurate location of the LED target area during subsequent detection and avoiding missed detections due to insignificant brightness differences. The contrast-enhanced image then undergoes denoising processing to eliminate useless information generated during image acquisition due to equipment noise, environmental interference, etc. Denoising processing must reduce the impact of noise on subsequent target detection while preserving the core features of the LED indicator, ensuring that key information such as the LED indicator's boundaries and brightness is not compromised, ultimately outputting an optimized enhanced image It′.

[0019] The core parameter for video stream acquisition is the sampling frame rate, which must strictly satisfy the Nyquist sampling theorem to ensure that the sampling frame rate is no less than twice the highest blinking frequency of the LED indicator. For example, if the highest blinking frequency of the LED indicator in the scene is 12Hz, the sampling frame rate must be no less than 24FPS to ensure that the on-off process of the LED indicator can be completely captured, providing accurate timing data support for subsequent blinking frequency calculation.

[0020] S102, based on the improved YOLOv8 lightweight architecture, inserts the CBAM attention module and adds a small target detection layer to detect dense LED targets and outputs bounding box coordinates and confidence scores. It calculates the mean V component of the HSV value of each detection box and determines the LED on / off state through dynamic thresholding.

[0021] In one implementation, video stream data from a dense LED inspection scenario is preprocessed to generate enhanced image data adapted to the improved YOLOv8 architecture. The video stream data includes images of dense LED arrays in the target industry's machine room, and the associated data consists of LED target detection annotation data and on / off state verification data. The video stream data focuses on machine room scenarios in industries such as railways, power, and oil and gas, specifically collecting continuous images of dense LED indicator arrays. It must clearly present the arrangement, spacing characteristics, and brightness variations of the LEDs to ensure accurate capture of brightness differences caused by manufacturing processes and signal strength, providing authentic and effective raw data support for subsequent target detection and state determination. The associated data includes LED target detection annotation data and on / off state verification data. The detection annotation data is used to train the detection accuracy of the improved YOLOv8 architecture and clarify the bounding box positions of the LED targets; the on / off state verification data is used to calibrate subsequent dynamic threshold determination results to ensure the accuracy of on / off state recognition.

[0022] The acquired video stream data undergoes preprocessing according to the following procedure: First, grayscale conversion is performed, transforming the 3D RGB color video frames into 1D grayscale images, removing redundant color information, and focusing on the core features of brightness differences between LEDs and the background. Next, contrast enhancement processing is implemented to address the issue of significant differences in brightness between different LEDs, strengthening the brightness characteristics of bright LEDs and the distinguishability of dark LEDs, avoiding missed detections due to insufficient brightness. Finally, noise reduction processing is performed to eliminate noise generated by equipment vibration and ambient light interference during image acquisition, ensuring that the boundary contours and brightness information of the LED indicators are not damaged. Through these three preprocessing steps, enhanced image data adapted to the improved YOLOv8 lightweight architecture is generated, meeting the architecture's quality requirements for input images and providing high-quality data input for subsequent CBAM attention module extraction of small target LED features and accurate localization of the small target detection layer (P2 layer).

[0023] The enhanced image data after adaptation is analyzed in a targeted manner to generate a list of core detection and judgment variables. These variables include the CBAM module parameters of the YOLOv8 backbone layer 9, the feature weights of the channel attention branches in the Neck layer, the detection parameters of the small target detection layer, the calculation window variable for the HSV value V component, the proportional coefficient k of the dynamic threshold, the median calculation variable for LED on / off determination, and the detection box confidence threshold variable. The core of the targeted analysis of the enhanced image data adapted to the improved YOLOv8 architecture is to extract the key variables supporting LED detection and on / off state determination, clarifying the definition, function, and value basis of each variable to ensure the accuracy and stability of subsequent detection processes. The specific details of each variable are as follows:

[0024] The CBAM module parameter variable in YOLOv8 backbone layer 9: This variable is directly related to the operating logic of the CBAM (Convolutional Block Attention Module) attention mechanism, and its core function is to adjust the weight distribution ratio between spatial attention and channel attention. For densely packed small LED targets, this parameter enhances the model's attention to LED features while suppressing interference from background environmental features, allowing the model to more accurately lock onto LED targets in complex and dense scenes, providing reliable feature support for subsequent bounding box localization.

[0025] The Neck layer is responsible for the core task of fusing features at different scales. This variable is used to optimize the fusion weights of features from each channel. In dense LED scenes, the features of different LED targets tend to overlap. By dynamically adjusting this variable, effective LED features can be highlighted, redundant features can be weakened, and the feature discrimination between different LED targets can be improved. This ensures that the fused features can accurately reflect the position and shape information of individual LEDs, providing high-quality feature input for the small target detection layer.

[0026] The detection parameters of the small target detection layer (P2 layer) include two core sub-parameters: the detection box size threshold and the initial confidence threshold. Considering the small size and dense arrangement of LED indicators, the detection box size threshold needs to be adapted to the characteristics of small targets and set to a range that closely matches the actual size of the LEDs to avoid multiple LEDs being misidentified as a single target due to an excessively large detection box. The initial confidence threshold is used to initially screen the detection results, eliminating low-confidence detection boxes that clearly do not belong to LEDs, reducing subsequent computational redundancy. Its value needs to balance the detection rate and false detection rate to adapt to the detection requirements in dense scenes.

[0027] The calculation window variable for the V component of the HSV value clarifies the calculation range of the V component (brightness) of pixels within the detection frame. Specifically, it extracts the V component and calculates the mean value only for pixels within the LED detection frame (rectangular area) output by YOLO. By limiting the calculation window, interference from background pixels outside the detection frame on the mean brightness can be avoided, ensuring that the calculated HSV value V component accurately reflects the LED's brightness state and provides accurate basic data for subsequent on / off determination.

[0028] The scaling factor k of the dynamic threshold is the core parameter for dynamically generating the LED on / off judgment threshold, with a preferred value of 0.3. Its function is to proportionally adjust the median of the average HSV values ​​of all LED detection frames to generate an adaptive threshold—ensuring that the threshold adapts to the overall brightness distribution of LEDs within the current frame, while also distinguishing the brightness boundaries of on / off LEDs through the scaling factor. This avoids misjudgments of on / off states caused by a single fixed threshold, and is particularly suitable for scenarios where different LEDs exhibit brightness differences due to manufacturing processes and signal strength.

[0029] The median operation variable for LED on / off determination corresponds to the median operation, the core function of which is to calculate the median of the average HSV value of all detected LEDs in frame t. Since some LEDs in dense scenes may exhibit abnormal brightness, taking the median avoids the influence of extreme values ​​on threshold generation, making the generated threshold more consistent with the brightness distribution of most LEDs and improving the overall accuracy of on / off determination.

[0030] The detection box confidence threshold variable is used for the final filtering of detection boxes output by YOLO, retaining only detection results with a confidence level (the probability that the box contains an LED, ranging from 0 to 1) higher than the threshold. Its value needs to be set according to the actual application scenario, ensuring that no real LED targets are missed while excluding invalid detection boxes with too low confidence, reducing unnecessary workload in subsequent HSV value calculation and state determination, and ensuring the overall efficiency of the process.

[0031] The enhanced image data, core detection and judgment variable list, original video stream data, and LED-related data are synchronously processed and accurately judged. First, the YOLOv8 architecture is improved by inserting a CBAM module and adding a P2 layer to detect dense LED targets and output bounding box coordinates and confidence scores. Then, the mean V component of the HSV value of each pixel within the detection box is calculated. The core of this synchronous processing of the enhanced image data, core detection and judgment variable list, original video stream data, and LED-related data is to achieve accurate localization of dense LED targets through an improved YOLOv8 architecture. Then, based on HSV values ​​and dynamic thresholds, the on / off state is determined, providing foundational data for subsequent processes. The details are as follows:

[0032] The lightweight YOLOv8 model is chosen as the basic architecture. Its lightweight nature reduces computational overhead, perfectly adapting to the limited hardware resources of intelligent inspection robots, ensuring real-time performance of the inspection process, and meeting the high-efficiency processing requirements of industrial inspection scenarios. A CBAM (Convolutional Block Attention Module) is inserted into layer 9 of the YOLOv8 backbone. This module works synergistically through channel attention and spatial attention. Channel attention strengthens the weights of key feature channels of the LED target, while spatial attention accurately focuses on the area where the LED is located, effectively suppressing background interference and significantly improving the feature extraction accuracy of small target LEDs in dense scenes.

[0033] A channel attention branch is embedded in the upsampling path of the Neck layer. The Neck layer, as the core of feature fusion at different scales, dynamically adjusts the fusion weights of features at each scale through this branch. This prioritizes retaining effective features of the LED target, weakens redundant information, optimizes feature transfer efficiency, and ensures that the fused features accurately reflect the position and shape of the dense LEDs. A small target detection layer (P2 layer) is added to specifically adapt to the detection needs of small-sized LEDs. Addressing the characteristics of small-sized and densely arranged LED indicators, the P2 layer compensates for the shortcomings of traditional architectures in small target detection through targeted detection of small-scale feature maps, significantly improving the detection coverage and localization accuracy of dense LEDs.

[0034] The preprocessed enhanced image data is input into the improved YOLOv8 architecture, with the core detection and decision variable lists participating in the computation simultaneously: the CBAM module parameter variables in the 9th layer of the YOLOv8 backbone adjust the attention weights, the feature weight variables of the attention branch in the Neck layer optimize feature fusion, and the detection parameter variables of the small target detection layer (P2 layer) adapt to small-size detection, ensuring the targeted and accurate nature of the model computation. The model output dimension is (Nt, 5), where Nt is the number of LEDs detected in frame t. Each detection target corresponds to 5 key parameters: accurately identifying the position of the LED in the image, providing raw data for subsequent coordinate transformation; defining the LED target range, ensuring that subsequent HSV value calculations focus only on the LED region; and confidence (range 0~1), representing the probability that the bounding box contains an LED, used for initial screening of reliable detection results and elimination of invalid detection boxes.

[0035] For each LED target bounding box (rectangular area), the HSV color space V component (luminance) of all pixels within the box is extracted. Combining the range defined by the calculation window variable of the HSV value V component, the total number of pixels within the box is counted. The average HSV value of the i-th detection box in the t-th frame is calculated using the mean formula, which accurately reflects the brightness state of the LED.

[0036] An adaptive threshold is generated based on the scaling factor k (preferably 0.3) and the median operation variable in the core detection and judgment variables. First, the average HSV value of all detected LEDs in frame t is calculated. Then, the median is calculated and multiplied by the scaling factor k to obtain the formula: Adaptive Threshold. median ,in, This is the adaptive threshold for the t-th frame. This threshold adapts to the overall brightness distribution of the LEDs in the current frame, avoiding misjudgments caused by a fixed threshold.

[0037] A confidence threshold is set, and detection boxes with confidence levels below the threshold are removed, retaining only reliable LED targets. This reduces the interference of invalid data on subsequent processes and ensures data quality. The average HSV value of each LED detection box is compared with an adaptive threshold. If the average HSV value is greater than the adaptive threshold, the LED is determined to be in a bright state; if it is less than or equal to the adaptive threshold, it is determined to be in a dark state. This rule can adapt to the brightness differences of different LEDs. Finally, the localization results (boundary box coordinates) and the on / off state determination results of dense LED targets are output, providing accurate and reliable basic data support for subsequent processes such as coordinate transformation, temporal clustering, and frequency calculation.

[0038] S103 converts the absolute coordinates of the LED into normalized relative coordinates, then discretizes them into adaptive grid relative coordinates. DBSCAN density clustering is used to perform spatial clustering within a single frame to construct a spatiotemporal correlation matrix. Combining position similarity and state weights, cross-frame temporal correlation clustering is performed to generate the temporal trajectory of the same physical LED.

[0039] In one implementation, based on the core distribution characteristics of dense LED target detection coordinates—dense arrangement, compact pixel-level distribution, and susceptibility to slight coordinate drift caused by lens shake—and the core cross-frame association rules in industrial inspection scenarios that require continuous maintenance of the spatiotemporal correlation of the same physical LED across frames and ensure the uniqueness of single-label multi-target tracking, this application establishes a bidirectional collaborative decision-making mechanism between the coordinate transformation module and the clustering analysis engine. Through parameter interaction, requirement matching, and strategy adaptation between modules, a targeted overall strategy for coordinate processing and clustering analysis is jointly determined. From the underlying coordinate expression and clustering dimension design level, the tracking failure problem caused by shake, resolution differences, and dense target overlap in dense LED scenarios is solved.

[0040] As the core of front-end data processing, the coordinate transformation module is responsible for outputting standardized and interference-resistant coordinate data. Based on the actual video stream resolution, LED array density, and measured lens shake data, it feeds back key information such as noise characteristics, distribution density, and cross-frame fluctuation range of the coordinate data to the clustering analysis engine. The clustering analysis engine, based on its own algorithm adaptability, computational efficiency requirements, and clustering accuracy goals, proposes specific requirements for coordinate expression, dimensional simplification, and noise suppression to the coordinate transformation module. After multiple rounds of parameter negotiation and strategy matching, the two modules clearly define a dual-core processing approach: normalization transformation to eliminate resolution differences and grid discretization to suppress shake noise. This addresses the resolution adaptation problem of different acquisition devices (720p / 1080p / 4K) and the coordinate drift noise problem caused by the movement of the inspection robot and the mechanical vibration of the camera.

[0041] Based on the core idea of ​​the aforementioned collaborative negotiation, a unified coordinate representation was ultimately determined to be a two-layer coordinate system consisting of normalized relative coordinates and grid relative coordinates. Normalized relative coordinates serve as the base layer, achieving dimensionless quantization and resolution independence of coordinate data. Grid relative coordinates serve as the optimization layer, achieving discretization of coordinate data and smoothing of jitter noise. The two-layer coordinate system supports each other, preserving the spatial accuracy of coordinates while improving their anti-interference capability. Simultaneously, the clustering analysis engine clearly focuses on two dimensions: spatial similarity and cross-frame temporal correlation. It abandons the limitation of traditional clustering relying solely on a single spatial location, deeply integrating intra-frame spatial clustering with cross-frame temporal correlation. This ensures that the clustering results not only conform to the spatial distribution characteristics of dense LEDs within a single frame but also meet the temporal tracking requirements of the same physical LED across frames. From a dimensional design perspective, this ensures that the overall strategy is fully adaptable to the needs of dense LED inspection scenarios in industries such as railways and power, laying a precise and stable coordinate data foundation for subsequent spatiotemporal joint clustering analysis.

[0042] After determining the core processing approach and coordinate representation, this application adopts a two-stage normalization-gridization transformation mechanism to standardize the LED absolute coordinate data throughout the entire process. The original pixel-level absolute coordinates are gradually converted into standardized coordinate data that is suitable for cluster analysis and has strong anti-shake capabilities. Each transformation stage is closely related to the core requirements of coordinate stability and anti-shake, ensuring that the transformed coordinate data not only meets the algorithm's computational efficiency but also accurately reflects the actual physical position of the LED, while effectively suppressing coordinate fluctuations caused by lens shake.

[0043] The first stage is normalized relative coordinate transformation. Using the actual pixel width (Wt) and height (Ht) of a single frame image as a reference, the core coordinates (x, y of the bounding box center point) of each LED target detection box are transformed using dimensionless quantization. The transformation formula is as follows: ,make , The value always remains within the range of [0,1]. The core function of this stage is to eliminate the coordinate scale differences caused by different acquisition devices and different video stream resolutions. For example, the absolute coordinates of an LED at the same physical location are (960,540) at a resolution of 1920×1080 and (640,360) at a resolution of 1280×720. After normalization, both are (0.5,0.5), achieving a unified expression of coordinate data at different resolutions. This allows subsequent clustering analysis to proceed without considering resolution differences, significantly improving the algorithm's versatility and adaptability. At the same time, the normalization transformation compresses large-scale pixel coordinates into small-range relative coordinates, effectively reducing the numerical scale of subsequent grid discretization and clustering calculations, improving the overall computational efficiency of the algorithm, and meeting the lightweight deployment requirements of intelligent inspection robots.

[0044] The second stage is adaptive grid relative coordinate discretization. Based on the normalized relative coordinates, the number of adaptive grids (Ng) is dynamically determined according to the actual density of the LED array. Then, the continuous normalized relative coordinates are discretized into discrete grid relative coordinates through floor operation. The conversion formula is as follows: The adaptive grid number (Ng) is not a fixed value, but is dynamically adjusted based on the number of LEDs detected within a single frame and the average pixel spacing between LEDs. In dense scenes (average LED spacing < 20 pixels), Ng is set to 80-100 to ensure sufficient grid resolution to distinguish adjacent LEDs. In sparse scenes (average LED spacing ≥ 20 pixels), Ng is set to 50-80 to further suppress jitter noise while maintaining distinguishability, achieving dynamic matching between grid resolution and LED density. The core function of this stage is to discretize the continuous coordinate space into finite grid units, so that the pixel-level small coordinate drift (usually 1-5 pixels) caused by lens shake is confined to the same grid unit. For example, if the normalized coordinates drift from 0.5123 to 0.5147, after discretization with Ng=50, the grid coordinates are all 25, effectively smoothing the coordinate noise caused by jitter. This allows the same physical LED to maintain the consistency of grid coordinates even under cross-frame jitter, providing a stable coordinate reference for subsequent spatial clustering.

[0045] In the normalization-gridification two-stage conversion process, each step revolves around the two core requirements of coordinate stability and anti-jitter. Normalization achieves coordinate stability and uniformity at the scale level, while grid discretization effectively suppresses jitter noise at the spatial level. The two complement each other and work synergistically. After the conversion, standardized coordinate data containing both normalized relative coordinates and grid relative coordinates is generated. This data retains the high-precision spatial position information of the normalized relative coordinates, which can be used for accurate calculation of cross-frame position similarity. It also has the high anti-jitter characteristics of the grid relative coordinates, which can be used for stable execution of spatial clustering within a single frame. At the same time, the numerical scale of the standardized coordinate data is significantly compressed, which meets the computational efficiency requirements of the clustering analysis engine. This provides high-quality coordinate data support for the efficient and accurate execution of DBSCAN density clustering and cross-frame temporal correlation clustering. It solves the coordinate instability problem caused by resolution differences and lens shake in dense LED scenes from the underlying data level, ensuring the continuity and uniqueness of cross-frame tracking of the same physical LED.

[0046] Based on the distribution characteristics and cross-frame association rules of dense LED target detection coordinate data, the coordinate transformation module and clustering analysis engine negotiate to determine the coordinate processing and clustering strategies. The coordinate representation and basic clustering dimensions are determined through normalization transformation and grid discretization. Based on the distribution characteristics of dense LED target detection coordinates—"densely arranged and susceptible to jitter"—and the association rule of "maintaining cross-frame association of the same physical LED," the coordinate transformation module and clustering analysis engine collaboratively negotiate the processing strategy. The core idea is clearly defined as "normalization transformation to eliminate resolution differences and grid discretization to suppress jitter noise," determining the coordinate representation as "normalized relative coordinates + grid relative coordinates." The basic clustering dimensions focus on spatial similarity and cross-frame temporal association to ensure adaptation to the needs of dense LED scenarios.

[0047] A two-stage transformation mechanism of normalization and gridding is adopted to process the absolute coordinate data of LEDs. First, normalized relative coordinates are calculated according to the image width and height. Then, based on the adaptive number of grids, the coordinates are discretized into grid relative coordinates. This process addresses the core requirements of coordinate stability and anti-jitter performance, generating standardized coordinate data. Based on the distribution characteristics of dense LED target detection coordinates—"densely arranged and susceptible to jitter"—and the association rule of "maintaining cross-frame correlation of the same physical LED," the coordinate transformation module and the clustering analysis engine collaboratively negotiate the processing strategy. The core idea is clearly defined as "normalization transformation to eliminate resolution differences and grid discretization to suppress jitter noise." The coordinate expression form is determined to be "normalized relative coordinates + grid relative coordinates." The basic dimensions of clustering focus on spatial similarity and cross-frame temporal correlation to ensure adaptability to the needs of dense LED scenarios.

[0048] This application performs validity checks on standardized coordinate data, initiates noise suppression processing for coordinate fluctuations caused by camera shake, triggers a grid parameter fine-tuning mechanism for accuracy deviations resulting from coordinate discretization, and performs a screening and removal process for outlier coordinate values. This generates an optimized coordinate data scheme that includes noise suppression measures, parameter fine-tuning strategies, and outlier removal methods. To further improve the quality of the coordinate data and ensure the accuracy and stability of subsequent cluster analysis, this application performs comprehensive validity checks on standardized coordinate data, initiates targeted processing mechanisms for different types of anomalies and deviations, and ultimately generates an optimized coordinate data scheme that includes noise suppression measures, parameter fine-tuning strategies, and outlier removal methods, providing highly reliable input data for cluster analysis.

[0049] To address minor coordinate fluctuations caused by camera shake (typically with a fluctuation range ≤ 3 pixels), the "fluctuation tolerance" characteristic of grid discretization is utilized to naturally suppress noise through the spatial inclusiveness of grid cells. Specifically, minor coordinate fluctuations within the same grid cell do not change the grid coordinate assignment of the LED, thus preventing such noise from interfering with the clustering results. For example, if an LED's normalized coordinates in three consecutive frames are (0.512, 0.321), (0.515, 0.323), and (0.513, 0.320), after grid discretization with Ng=50, the grid coordinates are all (25, 16), effectively filtering out the minor fluctuation noise caused by camera shake. Simultaneously, for coordinate data with slightly larger fluctuations but not exceeding the range of a single grid cell, no additional intervention is performed to ensure the authenticity and stability of the coordinate data and avoid positional deviations caused by excessive smoothing.

[0050] During coordinate verification, if excessive concentration or dispersion of coordinate data is detected (i.e., a single grid cell contains ≥5 LED targets, or a single LED target occupies ≥3 adjacent grid cells), a grid parameter (Ng) fine-tuning mechanism is triggered. This dynamically adjusts the number of grid cells based on the actual scene to ensure the grid resolution matches the LED distribution density. The specific fine-tuning rules are as follows: when excessive concentration of coordinates is detected (indicating LED density is higher than the current grid resolution's adaptability range), Ng is increased by 20%~30% to refine the grid cells and improve spatial discriminability; when excessive dispersion of coordinates is detected (indicating LED density is lower than the current grid resolution's adaptability range), Ng is decreased by 20%~30% to expand the grid cells and enhance anti-jitter capability. For example, if an initial Ng=60 indicates that multiple grid cells contain excessive LED counts, Ng is adjusted to 75, and the grid is re-discretized to control the number of LEDs in each grid cell to 1~3, ensuring the accuracy of cluster analysis.

[0051] For invalid coordinates outside the image range and outliers that deviate significantly from the coordinates of most LEDs (defined as outliers by calculating the Euclidean distance between the coordinates and the overall LED coordinate mean; a distance ≥ 3 times the standard deviation is considered an outlier), a rigorous screening and removal process is implemented. First, all standardized coordinate data is traversed to identify and mark invalid coordinates and outliers. Second, marked invalid coordinates are directly removed and not included in subsequent clustering analysis. For outliers, the confidence level of their corresponding LED detection boxes is further verified. If the confidence level ≥ 0.2, they are identified as potential edge LED targets, their coordinate data is retained, and they are recorded as special markers. If the confidence level < 0.2, they are considered invalid detection results and are removed. This process effectively avoids the impact of outlier coordinates on clustering accuracy, ensuring that all coordinate data participating in clustering are valid and reliable LED target coordinates.

[0052] By integrating noise suppression principles, grid fine-tuning ranges, and anomaly detection criteria, a standardized optimized coordinate data scheme is formed. This scheme clearly records key parameters of coordinate transformation (such as the final values ​​of Wt, Ht, and Ng), the applicable range of noise suppression, the fine-tuning thresholds and adjustment ranges of grid parameters, and the criteria and handling methods for anomaly coordinates. This ensures the traceability and consistency of coordinate data during subsequent clustering analysis, providing standardized data support for the stable execution of the clustering algorithm.

[0053] The optimized coordinate data scheme is integrated and executed. Intra-frame spatial clustering is completed using DBSCAN density clustering, and a spatiotemporal correlation matrix is ​​constructed simultaneously. Position similarity is calculated using a Gaussian kernel function, and cross-frame temporal correlation clustering is achieved by combining state weights, generating the temporal trajectory of the same physical LED. Based on the optimized coordinate data scheme, this application completes the clustering analysis and temporal trajectory generation of dense LEDs through a three-step process: "intra-frame spatial clustering - spatiotemporal correlation matrix construction - cross-frame temporal correlation clustering," achieving accurate tracking of the same physical LED.

[0054] The DBSCAN density clustering algorithm is employed, using relative grid coordinates as input and Chebyshev distance as the distance metric (high computational efficiency, resistant to rotational disturbances, and intuitively reflects the horizontal and vertical spatial characteristics of the grid). Core parameters are set as follows: neighborhood radius ϵ=2 (i.e., searching for other LED targets within two adjacent grid units of the current grid cell) and minimum number of points MinPts=1 (allowing individual LED targets to form independent clusters, avoiding omissions due to insufficient points). The specific clustering process is as follows: traversing the grid coordinates of all LEDs, using each coordinate as a core point, searching for all other coordinate points within its neighborhood radius. If the number of points in the neighborhood is greater than or equal to MinPts, these coordinate points are clustered into one cluster; otherwise, the core point forms a separate cluster. LED targets within the same cluster belong to the same physical location in space, thus achieving spatial differentiation of dense LEDs within a single frame, laying the foundation for subsequent cross-frame temporal correlation. For example, in a certain frame of an image, the grid coordinates of the LEDs are (25,16), (25,17), (26,16), and (30,20). After DBSCAN clustering, the first three coordinates are clustered into one cluster (they overlap within the neighborhood radius), and the last coordinate is in a separate cluster, which accurately distinguishes the LED targets in different physical locations.

[0055] After completing the spatial clustering within a single frame, an M×M spatiotemporal correlation matrix is ​​simultaneously constructed (where M is the total number of clusters appearing in all frames), and the matrix elements are... This represents the similarity score between the i-th cluster (from frame t) and the j-th cluster (from frame t+1) that belong to the same physical LED. The row indices of the matrix correspond to all clusters in the previous frame, and the column indices correspond to all clusters in the next frame. The scores of the matrix elements quantify the degree of association between clusters in different frames. Initially, all matrix elements have a score of 0. Subsequent cross-frame similarity calculations gradually accumulate the scores to form a complete spatiotemporal association matrix. This matrix comprehensively records the association relationships between clusters in different frames, providing a quantitative basis for cross-frame temporal association clustering and avoiding the subjectivity and bias of traditional association methods.

[0056] Cross-frame temporal association clustering achieves accurate association between different frame clusters through a process of "positional similarity calculation - state weight fusion - dynamic threshold filtering," generating temporal trajectories of the same physical LED. Positional similarity calculation uses a Gaussian kernel function to convert the Euclidean distance between clusters into a similarity score of 0 to 1, as shown in the formula. This means converting the Euclidean distance into a similarity of 0 to 1. Let be the normalized coordinates of the k-th point in the t-th frame; is the normalized coordinate of the l-th point in the (t + 1)-th frame; σ is the smoothing parameter (which can take values from 0.05 to 0.1). σ is inversely proportional to the grid resolution and positively correlated with the LED density: in sparse scenarios, σ = 0.05 (sensitive to coordinates and differentiating adjacent LEDs); in dense scenarios, σ = 0.10 (expanding the Gaussian kernel range and tolerating jitter). , where is the number of grids, is the average LED spacing.

[0057] State weight fusion: Calculate the state weight ω by combining the on / off state consistency of the LEDs. The formula is ω = 1 (when the on / off states of the LEDs corresponding to the two clusters are the same) or ω = α (when the on / off states of the LEDs corresponding to the two clusters are different), where α ∈ (0, 1) is the state inconsistency penalty factor, and the recommended value is 0.4. This value avoids too low scores when the state changes frequently due to LED blinking (likely to occur when α < 0.3) and prevents false associations when the LED states happen to be the same by coincidence (likely to occur when α > 0.5). Multiply the position similarity score by the state weight to obtain the comprehensive similarity score of the cluster pair, and accumulate it to the corresponding element in the spatio-temporal association matrix.

[0058] Considering the score accumulation difference caused by different video lengths (low scores for short videos and high scores for long videos), adopt a dynamic threshold strategy to screen valid associated clusters. The threshold formula is , where T is the total number of frames, is the average state weight of all processed frame pairs (using the arithmetic mean of all frame pairs, including cases with the same and different states, to ensure the adaptive nature of the threshold). If the element in the spatio-temporal association matrix is ≥ T, then it is determined that cluster i and cluster j belong to the same physical LED and they are associated; if < T, then it is determined that they are different physical LEDs and no association is made. Through multi-frame cumulative association, finally generate the complete temporal trajectory of each physical LED from the first frame to the last frame, realizing precise tracking and temporal information integration of dense LEDs.

[0059] S104. Process the on / off state sequence of the LEDs corresponding to the temporal trajectory. For frames with occasional detection failures, use linear interpolation to supplement the data and binarize it. Extract the rising edge of the state sequence through the zero-crossing detection method, and calculate the LED blinking frequency in combination with the video sampling frame rate.

[0060] In one implementation, a time-series data processing framework is used to preprocess the on / off state sequence corresponding to the LED timing trajectory. For occasional detection failure frames with a single frame YOLO confidence score less than 0.2 or within frame rate / 5 frames, missing data is supplemented through linear interpolation, and then binarization is performed at a threshold of 0.5 to generate a complete and regular binary state sequence. Based on the time-series data processing framework, to address the occasional detection failure problem in the on / off state sequence corresponding to the LED timing trajectory, a complete and regular binary state sequence is generated through linear interpolation and binarization of missing data, providing a standardized time-series data foundation for subsequent frequency calculations.

[0061] First, determine the type of failed detection frames. Frames with a single YOLO output box confidence score less than 0.2, or frames with a consecutive number of missing frames within the frame rate / 5 range, are classified as occasional failed detection frames, and their on / off state is recorded as a missing value. If the number of consecutive missing frames exceeds the frame rate / 5, it is classified as long-term occlusion, triggering a trajectory interruption alert, and is not included in this interpolation step. For occasional failed detection frames, linear interpolation is used to supplement the on / off state data, filling the gaps in the time sequence. The interpolated state data is then binarized with a threshold of 0.5, converting consecutive state values ​​into binary data of 0 (off) or 1 (on), generating a complete and regular binary state sequence.

[0062] The binary state sequence is associated with LED flicker characteristics, rising edge detection rules, and valid frame selection criteria to establish a correspondence between state transitions and rising edges, generating a rising edge candidate dataset. By associating the binary state sequence, LED flicker characteristics, rising edge detection rules, and valid frame selection criteria, a correspondence between state transitions and rising edges is established. Rising edge candidate frames that meet the preliminary conditions are selected from the binary state sequence to generate a rising edge candidate dataset. The LED flicker characteristics are clarified. LED flicker is a periodic transition between on and off states. The rising edge is the state transition process from off state (0) to on state (1), which is the core feature for calculating the flicker frequency. Only transitions from 0 to 1 in the binary state sequence are identified as rising edge candidates, excluding falling edges from 1 to 0, frames with no state change, and intermediate value transitions generated by interpolation. Invalid frames caused by lens shake and detection errors are removed, and only frames with the required detection coverage and the confidence level of state determination are retained for rising edge identification. Traverse the binary state sequence, identify all 0→1 state transition frames that meet the above rules and criteria, integrate their frame numbers and state information before and after the transition, and generate rising edge candidate dataset.

[0063] This step addresses the characteristics of high LED flicker frequency and rapid state transitions in industrial inspection scenarios such as railways and power plants. It binds rising edge detection to the actual flickering pattern of LEDs, avoiding misjudging invalid state fluctuations as rising edges, ensuring the validity of the candidate dataset, and laying the foundation for accurate frequency calculation in the future.

[0064] A zero-crossing detection model is constructed based on a candidate dataset. It uses a binary state sequence as input, edge transition determination as the core logic, and valid rising edge selection rules as the basis for judgment. This extracts a set of valid rising edge moments and generates rising edge feature data. The edge transition determination is performed by calculating the inter-frame difference of the state sequence. Determine the state transition type: Treat the state sequence Sm as a binary signal and detect edge transitions: , =+1: This indicates a rising edge (from 0 to 1); =-1: This indicates a falling edge (from 1 to 0); =0: Determined as no change in state, output the set of valid rising edge times of the LED. .

[0065] Further filtering is performed on the candidate rising edge dataset, retaining only rising edges that meet all of the following conditions as valid rising edges: the LED state corresponding to the rising edge transitions strictly from 0 to 1, excluding false transitions caused by interpolation-generated intermediate values; the minimum pulse width of the LED bright state is greater than 2 frames, avoiding misjudging short-term brightness fluctuations as valid flickering; the frame interval between two adjacent valid rising edges is greater than 3 frames, excluding continuous false transitions caused by high-frequency noise.

[0066] Combining the video sampling frame rate, the frame interval between the first and last rising edges, and the number of valid rising edges, the frequency is calculated using the formula. The process involves calculating the flicker frequency, defining the LED flicker period and frequency values, and generating an LED frequency feature set that includes the flicker frequency and rising edge distribution. By combining key parameters such as the video sampling frame rate and the set of effective rising edge times, the LED flicker frequency is calculated using a frequency calculation formula, defining the flicker period and frequency values, and ultimately generating an LED frequency feature set that includes the flicker frequency and rising edge distribution.

[0067] The formula for flicker frequency is: (Hz), where, The number of valid rising edges (cycles); The frame interval from the first rising edge to the last rising edge; The video sampling frame rate must satisfy the Nyquist sampling theorem, meaning the sampling frame rate must be no less than twice the highest flicker frequency of the LED. For example, if the highest flicker frequency in the scene is 12Hz... (≥24 FPS) Substitute the above parameters into the flicker frequency formula to calculate the flicker frequency of the LED. The flicker period T is obtained through the reciprocal relationship. By integrating information such as flicker frequency, flicker period, effective rising edge time set, and rising edge frame interval distribution, an LED frequency feature set is generated, providing core data for subsequent state determination and confidence assessment.

[0068] S105 constructs a two-factor confidence evaluation model based on the mean and standard deviation of detection coverage and flicker interval to quantify tracking reliability and determine the LED's always-on / always-off state and tracking stability.

[0069] In one implementation, this model, to quantify the reliability of the dense LED flicker frequency tracking process, uses two core factors for comprehensive evaluation: detection coverage and the periodic stability of the flicker interval. The two factors are multiplied to obtain the final confidence value, which reflects the effectiveness of LED target tracking and the accuracy of flicker frequency calculation. The core formula of the model is as follows: ,in, This is the confidence score for individual LED m, ranging from (0,1]. A value closer to 1 indicates higher tracking reliability. To effectively detect the number of frames, , The mean and variance of the flash intervals; detection coverage. The percentage of frames in which LEDs were successfully detected; average interval frames. ; Interval frame number variance For example, in a 30-frame video, if LED_A is detected for 28 frames, then the coverage detection rate is 28 / 30 = 0.933.

[0070] Industrial applications such as railways and power plants have extremely high requirements for the reliability and accuracy of LED status detection. Low detection coverage means that there are many failures in LED positioning or identification, and poor periodic stability indicates that there are large errors in the calculation of flicker frequency. Dual-factor evaluation can comprehensively cover the two core requirements of "detection effectiveness" and "frequency calculation accuracy", avoiding the one-sidedness of single-factor evaluation.

[0071] Using both flicker frequency and confidence level as dual criteria, and combined with the safety level requirements of industrial inspection scenarios, key thresholds are set to determine the LED constant-on / constant-off state and tracking stability. The determination results directly serve the intelligent inspection fault early warning of mechanical room equipment in industries such as railways and power.

[0072] Step 1: Calculate the flicker frequency of individual LEDs Calculate the flicker frequency based on the number of valid rising edges and the frame interval: ,in, The number of valid rising edges (cycles); The frame interval from the first rising edge to the last rising edge; This refers to the video sampling frame rate.

[0073] Step 2: Determine the LED's always-on / always-off state and set the static frequency threshold. =0.5Hz. This threshold is determined based on the persistence of vision of the human eye and the "Standard and Specification for Flashing Mode of Industrial LED Signal Lights": When the LED state change cycle is >2 seconds (frequency <0.5Hz), the human eye can clearly distinguish between on and off, and no longer forms a "flickering" visual effect, which is defined as constant on / constant off.

[0074] If the calculated flicker frequency < (i.e., <0.5Hz), the LED is determined to be in a constantly on / constantly off state; if If the frequency is ≥0.5Hz, the LED is determined to be in a flashing state, and the specific flashing frequency value is output.

[0075] Step 3: Determine the stability of LED tracking and set a confidence threshold. The threshold value is adjusted according to the security level of the application scenario: High security level scenarios (railway signaling equipment, power room): =0.8; Typical monitoring scenarios: =0.6.

[0076] The judgment rule is to compare the confidence value Cm calculated by the model with the scene corresponding to... Comparison: If ≥ If the LED tracking is stable and the blinking frequency / on / off state determination result is valid, it can be used as a basis for equipment inspection; if < If the LED tracking is unstable, the blinking frequency / on / off state determination result is invalid, and staff need to be reminded to retest or manually verify it.

[0077] All parameters in this model are set based on actual industrial inspection scenarios, with no redundant parameters. Key thresholds can be flexibly adjusted according to scenario requirements. The core parameters and their values / setting criteria are as follows: Effective detection frame count threshold: Detection frame confidence ≥ 0.2, consistent with the confidence screening standard for LED target detection, ensuring data consistency. Static frequency threshold: =0.5Hz, set according to the persistence of vision of the human eye and industrial LED signal light standards, which is the industry-standard judgment standard; confidence threshold: =0.8 (high security level) / 0.6 (general scenario), set according to the reliability requirements of equipment inspection in industries such as railways and power. The threshold is increased in high security level scenarios to ensure the accuracy of the results. Valid rising edge screening criteria: minimum pulse width > 2 frames, adjacent interval > 3 frames, strict 0→1 transition, eliminating false transitions caused by noise and interpolation, ensuring the validity of the flicker interval calculation, and thus improving the accuracy of the period stability factor.

[0078] S106 uses the Lucas-Kanade sparse optical flow method or the Farneback dense optical flow method to estimate the global motion vector, and performs cumulative jitter compensation through dynamic ROI adjustment and coordinate drift correction model, outputting dense LED flicker frequency, state determination results and confidence evaluation results.

[0079] In one implementation, an appropriate optical flow method is selected based on the hardware computing power. Image feature points are extracted using Shi-Tomasi corner detection, and local motion interference is removed through median filtering to estimate the global motion vector. Specifically, the Lucas-Kanade sparse optical flow method is used on low-power platforms, while the Farneback dense optical flow method is used on GPU-accelerated platforms. Selecting the appropriate optical flow method based on the hardware computing power, combining it with Shi-Tomasi corner detection to extract feature points, and estimating the global motion vector after removing local motion interference provides core data support for subsequent jitter compensation, thus adapting to the lightweight deployment requirements of intelligent inspection robots with different hardware configurations.

[0080] Step 1: Optical flow method adaptation selection. Considering the differences in computing power among industrial inspection robots, a "computing power tiered adaptation" strategy is adopted to ensure that the optical flow method meets jitter compensation accuracy requirements without exceeding the hardware's computing capabilities, thus guaranteeing real-time performance.

[0081] For low-computing-power platforms (such as embedded devices without GPU acceleration): choose the Lucas-Kanade sparse optical flow method. This method only processes key feature points, has low computational cost and low memory usage, and is suitable for lightweight deployment requirements.

[0082] GPU-accelerated platforms (such as inspection equipment equipped with a dedicated graphics card): Choose the Farneback dense optical flow method, which can cover all pixels of the image, provide more comprehensive motion vector estimation, and improve jitter compensation accuracy.

[0083] For the Lucas-Kanade sparse optical flow method (low-computing platform): Window size: Pixels (balancing local smoothing and edge preservation); Number of iterations: (Trade-off between convergence accuracy and time consumption); Number of pyramid levels: (For larger displacements, use a scaling factor of 0.5); Minimum eigenvalue threshold: (Remove weak texture points).

[0084] For Farneback dense optical flow (GPU-accelerated platform): Polynomial order: d=2 (using a quadratic polynomial to fit local brightness changes, adapting to the brightness distribution of the LED region); Neighborhood window size: (Spatial support of the fitted polynomial ensures complete capture of motion features); Gaussian kernel standard deviation: (Smooth polynomial coefficients to reduce noise interference); Number of pyramid layers: L=4, scaling factor s=0.5 (to handle displacement changes at different scales); Number of iterations: Titer=5 per pyramid layer (to ensure the convergence accuracy of each iteration).

[0085] Step 2: Shi-Tomasi corner detection extracts feature points. Corners are the intersections of images with drastic brightness changes or edge contours, and can stably reflect the motion characteristics of the image. Shi-Tomasi corner detection extracts feature points from the preprocessed enhanced image, providing a core tracking target for optical flow calculation of motion vectors. Furthermore, the feature points are mainly distributed in the LED array and device contour areas, ensuring a strong correlation between motion estimation and LED positioning.

[0086] Maximum number of corner points: set to 500 (select a maximum of 500 corner points from the entire image to balance computational load and feature point coverage); Quality level q=0.01 (only retain pixels with corner response values ​​greater than 1% of the maximum response value to filter high-confidence feature points); Minimum Euclidean distance: set to 10 pixels (the distance between any two selected corner points must be ≥10 pixels to avoid excessive feature point density leading to computational redundancy; if it is less than 10 pixels, corner points with higher response values ​​will be retained).

[0087] Step 3: Median filtering eliminates local motion interference. In industrial inspection scenarios, images may contain local motion interference (such as dust reflections on equipment surfaces, movement of small debris, etc.). This interference can cause the motion vectors of some feature points to be unrelated to the overall camera shake. Directly incorporating these into the calculation would affect the accuracy of the global motion vector. Median filtering processes the motion vectors of all feature points, eliminating vector data of abnormal local motion while retaining motion vectors that reflect the overall shake, ensuring the reliability of the global motion estimation.

[0088] Localized motion interference can cause false drift in LED positioning coordinates, affecting the tracking stability of densely packed LEDs. Median filtering can effectively eliminate such interference, ensuring that the global motion vector accurately reflects camera shake, providing an accurate basis for subsequent LED coordinate correction, and adapting to the high requirements for LED positioning accuracy in scenarios such as railways and power grids.

[0089] Step 4: Global motion vector estimation. Based on the selected feature points, the motion vector of each feature point is calculated using the selected optical flow method (Lucas-Kanade or Farneback). Combined with the result after median filtering, the global motion vector Vt of the image is determined. This vector contains the direction of motion and the magnitude of displacement, quantifying the overall motion state of the camera caused by shaking.

[0090] Based on global motion vectors and frame intervals, dynamic ROI adjustment is performed on the region containing the LED array in the image. Irrelevant backgrounds are eliminated to reduce computational overhead, generating a region of interest image focused on the LED target. The ROI of the LED array is adjusted based on global motion vectors and frame intervals, eliminating irrelevant backgrounds to reduce subsequent computational overhead and ensuring that computing power is concentrated on the LED target area, improving the real-time performance of the overall process. Using the previous YOLO target detection results, the rectangular region containing the LED array in the image is located as the initial ROI. To reduce computation, irrelevant backgrounds are eliminated, and the ROI is adjusted based on the motion vectors, i.e., the rectangular region where the LED array appears in the image. All subsequent calculations are performed only on this region. The core formula is as follows: , This is a geometric translation operation; The frame time interval; This is a global motion vector. In dense LED inspection scenarios, the background of the image occupies a large proportion (such as the walls of the server room, equipment casings, etc.), and these areas do not need to participate in LED detection and tracking. Dynamic ROI adjustment can reduce the calculation range to the LED array area, reducing the amount of computation by more than 60%, effectively adapting to the lightweight computing power requirements of intelligent inspection robots and ensuring real-time tracking performance.

[0091] A cumulative error correction model is constructed, calculating the cumulative displacement from frame 1 to the current frame using a correction intensity coefficient β. This model performs frame-by-frame drift correction on the LED target coordinates, preventing the accumulation of positioning errors caused by jitter. In industrial inspection scenarios, the mechanical vibrations generated during the movement of intelligent inspection robots cause continuous camera jitter. This jitter-induced coordinate drift exhibits significant cumulative characteristics—small displacements within a single frame, if not corrected in time, will amplify with increasing frame count, eventually leading to chaotic LED target positioning and severely affecting the continuity of cross-frame tracking and the accuracy of flicker frequency calculation. To address this issue, this application constructs a cumulative error correction model. By calculating the cumulative displacement frame-by-frame and dynamically adjusting the LED target coordinates, the cumulative positioning error caused by jitter is suppressed. The core idea of ​​the cumulative error correction model is to calculate the total camera displacement from frame 1 to the current frame based on the global motion vector estimated by the optical flow method. Then, the coordinate correction amplitude is dynamically adjusted using the correction intensity coefficient β, eliminating cumulative errors and avoiding coordinate jumps caused by overcorrection, ensuring long-term stability of the LED target coordinates. By subtracting the cumulative displacement from frame 1 to frame t from the β coefficient, and retaining the coordinates relative to the initial position, overcorrection is avoided and coordinate jumps are prevented. A cumulative error correction model is then established. β∈[0.1,0.3] is the correction intensity coefficient; This indicates the total amount the camera has moved from frame 1 to frame t. The greater the jitter, the larger β should be to quickly eliminate accumulated errors; however, an upper limit of 0.3 is set to prevent overcorrection from introducing oscillations.

[0092] The value of the correction intensity coefficient β is positively correlated with the jitter level of the inspection scene and needs to be dynamically adjusted according to the actual application scenario: For high jitter scenarios (such as the movement of a railway machine room inspection robot, or movement without a fixed track): β is set to 0.2~0.3 to quickly eliminate accumulated errors and avoid excessive amplification of coordinate drift. For low jitter scenarios (such as fixed track inspection of a power machine room, or stationary robot shooting): β is set to 0.1~0.2 for gentle correction, preventing coordinate fluctuations caused by overcorrection. The upper limit of β is set to 0.3, the core purpose of which is to avoid oscillations introduced by overcorrection—if β is too large, it may cause the LED coordinates to jump in opposite directions between adjacent frames, disrupting the tracking continuity. The accumulated displacement is obtained by accumulating the global motion vector frame by frame, where the global motion vector is the motion vector of each frame (estimated by optical flow and optimized by median filtering). This calculation process is synchronized in real time with the video frame processing flow, and the accumulated displacement is updated once after processing each frame to ensure the real-time nature of the correction.

[0093] Coordinate drift correction is performed frame by frame in the following steps to ensure the accuracy of the LED target coordinates in each frame: obtain the original grid relative coordinates of the LED target in frame t, and the global motion vector sequence from frame 1 to frame t; calculate the total displacement of the camera from the initial frame to frame t based on the cumulative displacement and the correction intensity coefficient β; transfer the corrected coordinate data to the subsequent modules, and save the cumulative displacement result of the current frame to provide a basis for the correction of the next frame.

[0094] Through the above process, minute coordinate drifts within a single frame are corrected in real time, and cross-frame cumulative errors are controlled to a very small range (usually ≤1 grid unit), ensuring that the coordinates of the same physical LED remain stable in a long video sequence, providing accurate timing trajectory data for subsequent flicker frequency calculation.

[0095] This application integrates LED positioning data after jitter compensation, flicker frequency calculation results, and two-factor confidence assessment values, and combines these with state determination rules to output the flicker frequency, state determination results, and confidence assessment results of dense LEDs. After jitter compensation, flicker frequency calculation, and confidence assessment, this application integrates the output data from each module and combines them with state determination rules to generate standardized dense LED detection results, providing a directly usable decision-making basis for industrial inspection.

[0096] The integrated core data comprises three main categories to ensure the completeness and reliability of the results, specifically: Positioning and jitter compensation data: relative coordinates of the LED target grid after cumulative error correction, and normalized relative coordinates, clearly defining the precise position of the LED in the image. Flicker frequency data: flicker frequency value (Hz), flicker period, and set of effective rise times for each individual LED, reflecting the flicker characteristics of the LED. Confidence and stability data: two-factor confidence assessment values, detection coverage, and the mean and standard deviation of flicker intervals, quantifying the reliability of tracking and the accuracy of frequency calculation.

[0097] Based on integrated data, a dual-state determination is performed on each individual LED to ensure the results meet the safety requirements of industrial inspection scenarios. For the constantly on / constantly off state determination: a static frequency threshold is set. =0.5Hz (Based on the persistence of vision of the human eye and the "Standard and Specification for Flashing Modes of Industrial LED Signal Lights": When the LED state change period is >2 seconds, the human eye can clearly distinguish between on and off, and no longer forms a "flickering" visual effect); if the LED's flicker frequency < (i.e., <0.5Hz), determined to be in a constantly lit / constantly off state; if If the frequency is ≥0.5Hz, it is determined to be a flickering state, and the specific frequency value is output.

[0098] Set confidence threshold Adjustments are made based on the security level of the application scenario. For high-security scenarios (railway signaling equipment, power control rooms): =0.8, rigorously filtering for reliable results; for general monitoring scenarios: =0.6, balancing reliability and detection coverage; if the two-factor confidence score is... ≥ Marked as "Stable Tracking," the result is valid and can be used as a basis for equipment inspection; if < If the result is marked as "unstable tracking", it is invalid and staff should be reminded to retest or manually verify it.

[0099] like Figure 2 As shown, a real-time interpretation system for dense LED flicker frequency integrating jitter compensation and confidence assessment is disclosed. The system includes:

[0100] The video stream preprocessing module 201 is used to acquire continuous video stream data, optimize the original video frames through grayscale processing, contrast enhancement technology and denoising algorithm, generate an enhanced image dataset adapted to subsequent detection, and provide high-quality input for LED target recognition.

[0101] The dense LED detection and state determination module 202 is based on the improved YOLOv8 lightweight architecture, inserts the CBAM attention module and adds a small target detection layer, accurately detects dense LED targets and outputs the bounding box coordinates and confidence, calculates the mean V component of the HSV value in the detection box, and outputs the LED on / off state in combination with the dynamic threshold determination rule.

[0102] The coordinate transformation and temporal clustering module 203 is used to convert the absolute coordinates of the LED into normalized relative coordinates, and then discretize them into adaptive grid relative coordinates. It completes spatial clustering within a single frame through DBSCAN density clustering, constructs a spatiotemporal correlation matrix, and combines position similarity and state weight to achieve cross-frame temporal correlation clustering, generating the temporal trajectory of the same physical LED.

[0103] The blinking frequency calculation module 204 is used to process the LED on / off state sequence. For frames that occasionally fail to be detected, linear interpolation is used to complete the data and binarization is performed. The rising edge of the state sequence is extracted by zero-crossing detection method, and the LED blinking frequency is calculated by combining the video sampling frame rate.

[0104] The confidence assessment and state determination module 205 is used to construct a two-factor confidence assessment model based on the mean and standard deviation of the detection coverage and flicker interval, quantify the tracking reliability, and determine the LED constant-on / constant-off state and tracking stability based on the frequency threshold.

[0105] The jitter compensation and result output module 206 is used to select the Lucas-Kanade sparse optical flow method or the Farneback dense optical flow method to estimate the global motion vector based on the hardware computing power. It achieves cumulative jitter compensation through dynamic ROI adjustment and coordinate drift correction model, and finally outputs the dense LED flicker frequency, state judgment result and confidence evaluation result.

[0106] A computing device includes a memory for storing computer program instructions and a processor for executing the computer program instructions, wherein when the computer program instructions are executed by the processor, the device is triggered to execute any one of the methods for real-time interpretation of dense LED flicker frequency that integrates jitter compensation and confidence assessment.

[0107] The methods and / or embodiments in this application can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. When the computer program is executed by a processing unit, it performs the functions defined in the methods of this application.

[0108] It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device.

[0109] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0110] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that this application can be implemented in other specific forms without departing from the spirit or essential characteristics of this application. Therefore, the embodiments should be regarded as exemplary and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within this application.

Claims

1. A method for real-time interpretation of dense LED flicker frequency integrating jitter compensation and confidence assessment, characterized in that, include: Acquire continuous video stream data, perform grayscale conversion, contrast enhancement, and noise reduction preprocessing to obtain an enhanced image; Based on the improved YOLOv8 lightweight architecture, the CBAM attention module is inserted and a small target detection layer is added to detect dense LED targets and output bounding box coordinates and confidence scores. The mean V component of the HSV value of each detection box is calculated, and the LED on / off state is determined by a dynamic threshold. The absolute coordinates of the LED are converted into normalized relative coordinates, and then discretized into adaptive grid relative coordinates. DBSCAN density clustering is used to perform spatial clustering within a single frame to construct a spatiotemporal correlation matrix. Cross-frame temporal correlation clustering is performed by combining position similarity and state weight to generate the temporal trajectory of the same physical LED. The LED on / off state sequence corresponding to the time trajectory is processed. For frames that occasionally fail to be detected, linear interpolation is used to supplement the data and binarize it. The rising edge of the state sequence is extracted by the zero-crossing detection method, and the LED flicker frequency is calculated by combining the video sampling frame rate. A two-factor confidence assessment model is constructed based on the mean and standard deviation of detection coverage and flicker interval to quantify tracking reliability and determine the LED constant-on / constant-off state and tracking stability. The global motion vector is estimated using the Lucas-Kanade sparse optical flow method or the Farneback dense optical flow method. Accumulated jitter compensation is performed through dynamic ROI adjustment and coordinate drift correction model. The output includes dense LED flicker frequency, state determination results, and confidence evaluation results.

2. The method for real-time interpretation of dense LED flicker frequency integrating jitter compensation and confidence assessment according to claim 1, characterized in that, Based on the improved YOLOv8 lightweight architecture, a CBAM attention module is inserted and a small target detection layer is added to detect dense LED targets and output bounding box coordinates and confidence scores. The mean V component of the HSV value of each detection box is calculated, and the LED on / off state is determined by a dynamic threshold, including: The video stream data of the dense LED inspection scenario is preprocessed to generate enhanced image data adapted to the improved YOLOv8 architecture. The video stream data covers dense LED array images of the target industry's mechanical room equipment, and the associated data are LED target detection annotation data and on / off state verification data. The adapted enhanced image data is analyzed in a targeted manner to generate a list of core detection and judgment variables, including the CBAM module parameter variables of the 9th layer of YOLOv8backbone, the feature weight variables of the channel attention branch of the Neck layer, the detection parameter variables of the small target detection layer, the calculation window variables of the HSV value V component, the proportional coefficient k variable of the dynamic threshold, the median calculation variable of the LED on / off judgment, and the detection box confidence threshold variable. The adapted enhanced image data, core detection and judgment variable list, original video stream data and LED-related data are processed synchronously and accurately judged. First, the CBAM module is inserted and the P2 layer is added by improving the YOLOv8 architecture to detect dense LED targets and output the bounding box coordinates and confidence scores. Then, the mean V component of the HSV value of each pixel in the detection box is calculated. Subsequently, an adaptive threshold is dynamically generated based on the proportional coefficient k and the median. Finally, by combining confidence filtering and on / off determination rules, the positioning results and on / off status determination results of dense LED targets are output.

3. The method for real-time interpretation of dense LED flicker frequency integrating jitter compensation and confidence assessment according to claim 1, characterized in that, The absolute coordinates of the LEDs are converted to normalized relative coordinates, then discretized into adaptive grid relative coordinates. DBSCAN density clustering is used for intra-frame spatial clustering to construct a spatiotemporal correlation matrix. Cross-frame temporal correlation clustering is then performed by combining position similarity and state weights to generate the temporal trajectory of the same physical LED, including: Based on the distribution characteristics and cross-frame association rules of dense LED target detection coordinate data, the coordinate transformation module and the clustering analysis engine negotiate to determine the coordinate processing and clustering strategy, and determine the coordinate expression form and basic clustering dimension through normalization transformation and grid discretization. A two-stage transformation mechanism of normalization-gridization is adopted to process the absolute coordinate data of LEDs. First, the normalized relative coordinates are calculated according to the image width and height. Then, the relative coordinates are discretized into grid relative coordinates based on the adaptive grid number. The core requirements of coordinate stability and anti-jitter are associated to generate standardized coordinate data. The standardized coordinate data is validated for validity. Noise suppression is initiated for coordinate fluctuations caused by camera shake. A mesh parameter fine-tuning mechanism is triggered for accuracy deviations caused by coordinate discretization. An abnormal coordinate value is filtered and removed. An optimized coordinate data scheme is generated, which includes a noise suppression scheme, a parameter fine-tuning strategy, and an anomaly removal method. The optimized coordinate data scheme is integrated and executed. Intra-frame spatial clustering is completed by combining DBSCAN density clustering, and a spatiotemporal correlation matrix is ​​constructed simultaneously. Position similarity is calculated by Gaussian kernel function, and cross-frame temporal correlation clustering is achieved by combining state weights to generate the temporal trajectory of the same physical LED.

4. The method for real-time interpretation of dense LED flicker frequency integrating jitter compensation and confidence assessment according to claim 3, characterized in that, The LED on / off state sequence corresponding to the time-series trajectory is processed. For frames that occasionally fail to be detected, linear interpolation is used to supplement the data and binarize it. The rising edge of the state sequence is extracted using the zero-crossing detection method. The LED flicker frequency is calculated by combining the video sampling frame rate, including: A time-series data processing framework is used to preprocess the on / off state sequence corresponding to the LED time-series trajectory. For occasional failed detection frames with a single frame YOLO confidence of less than 0.2 or within frame rate / 5 frames, missing data is supplemented by linear interpolation, and then binarization is performed at a threshold of 0.5 to generate a complete and regular binary state sequence. The binary state sequence is correlated with LED blinking characteristics, rising edge detection rules and effective frame selection criteria to establish the correspondence between state transitions and rising edges, and to generate a rising edge candidate dataset. A zero-crossing detection model is constructed based on the candidate dataset. The model takes a binary state sequence as input, edge transition judgment as the core logic, and effective rising edge selection rules as the judgment basis. The set of effective rising edge moments is extracted to generate rising edge feature data. Combining the video sampling frame rate, the frame interval between the first and last rising edges, and the number of valid rising edges, the frequency is calculated using the formula. Complete the flicker frequency calculation, where, The number of valid rising edges (cycles); The frame interval from the first rising edge to the last rising edge; To determine the video sampling frame rate, the LED flicker period and frequency values ​​are defined, and an LED frequency feature set containing flicker frequency and rising edge distribution is generated.

5. The method for real-time interpretation of dense LED flicker frequency integrating jitter compensation and confidence assessment according to claim 1, characterized in that, The global motion vector is estimated using either the Lucas-Kanade sparse optical flow method or the Farneback dense optical flow method. Accumulated jitter compensation is performed through dynamic ROI adjustment and coordinate drift correction models. The output includes the dense LED flicker frequency, state determination results, and confidence assessment results, including: The appropriate optical flow method is selected based on the hardware computing power. Image feature points are extracted by combining Shi-Tomasi corner detection. Local motion interference is removed by median filtering, and the global motion vector is estimated. The low computing power platform adopts the Lucas-Kanade sparse optical flow method, while the GPU acceleration platform adopts the Farneback dense optical flow method. Based on the global motion vector and frame time interval, dynamic ROI adjustment is performed on the region where the LED array is located in the image to eliminate irrelevant backgrounds to reduce computational overhead and generate a region of interest image that focuses on the LED target. A cumulative error correction model is constructed, and the cumulative displacement from the first frame to the current frame is calculated according to the correction intensity coefficient β. The LED target coordinates are then corrected frame by frame to avoid the accumulation of positioning deviation caused by jitter. The system integrates LED positioning data after jitter compensation, flicker frequency calculation results, and two-factor confidence evaluation values, and combines these with state determination rules to output the flicker frequency, state determination results, and confidence evaluation results of dense LEDs.

6. A real-time interpretation system for dense LED flicker frequency integrating jitter compensation and confidence assessment, characterized in that, The system includes: The video stream preprocessing module is used to acquire continuous video stream data, optimize the original video frames through grayscale processing, contrast enhancement technology and noise reduction algorithm, and generate an enhanced image dataset adapted for subsequent detection, providing high-quality input for LED target recognition; The dense LED detection and state determination module is based on the improved YOLOv8 lightweight architecture, inserts the CBAM attention module and adds a small target detection layer, accurately detects dense LED targets and outputs bounding box coordinates and confidence, calculates the mean V component of HSV value in the detection box, and outputs the LED on / off state in combination with dynamic threshold determination rules. The coordinate transformation and temporal clustering module is used to convert the absolute coordinates of LEDs into normalized relative coordinates, and then discretize them into adaptive grid relative coordinates. It completes spatial clustering within a single frame through DBSCAN density clustering, constructs a spatiotemporal correlation matrix, and combines position similarity and state weight to achieve cross-frame temporal correlation clustering, generating the temporal trajectory of the same physical LED. The blink frequency calculation module is used to process the LED on / off state sequence. For frames that occasionally fail to be detected, linear interpolation is used to complete the data and binarization is performed. The rising edge of the state sequence is extracted by zero-crossing detection method, and the LED blink frequency is calculated by combining the video sampling frame rate. The confidence assessment and state determination module is used to construct a two-factor confidence assessment model based on the mean and standard deviation of detection coverage and flicker interval, quantify tracking reliability, and determine the LED constant-on / constant-off state and tracking stability based on the frequency threshold. The jitter compensation and result output module is used to select either the Lucas-Kanade sparse optical flow method or the Farneback dense optical flow method to estimate the global motion vector based on the hardware computing power. It achieves cumulative jitter compensation through dynamic ROI adjustment and coordinate drift correction model, and finally outputs the dense LED flicker frequency, state determination result and confidence evaluation result.

7. An electronic device, characterized in that, include: First processor; and memory for storing executable instructions of the first processor; The first processor is configured to execute the real-time interpretation method for dense LED flicker frequency integrating jitter compensation and confidence assessment as described in any one of claims 1 to 5 by executing the executable instructions.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the second processor, it implements the method for real-time interpretation of dense LED flicker frequency that integrates jitter compensation and confidence assessment as described in any one of claims 1 to 5.

Images