Electric spark flicker real-time visual detection method based on regional integration algorithm

Abstract

The invention discloses an electric spark flicker real-time visual detection method based on a regional integration algorithm. The method comprises the following steps: S1, collecting a video data stream, and carrying out preprocessing and parameter setting; s2, performing real-time target detection based on a YOLO deep learning network, and positioning an ROI region; s3, performing image enhancement on the ROI by using a multi-parameter combination optimization technology; s4, spark feature extraction is carried out based on a regional brightness integral algorithm; s5, performing data processing and analysis by adopting a double-thread parallel architecture; s6, spark feature recognition is realized in combination with a dynamic threshold mechanism; and S7, real-time output and exception handling of a detection result are realized. According to the electric spark flicker real-time visual detection method based on the regional integration algorithm, environmental interference is effectively restrained, the processing efficiency is remarkably improved, and the method can be applied to scenes such as quality detection in the industrial production process, equipment operation state monitoring and online detection of an automatic production line.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and industrial automation detection, and in particular to a real-time visual detection method for electric spark flicker based on a regional integration algorithm. Background Art

[0002] With the rapid development of industrial automation and intelligent manufacturing, machine vision-based automatic inspection technologies have gained widespread application in the industrial sector. Real-time flicker detection is crucial for product quality control and equipment status monitoring in numerous production scenarios. Examples include LED performance testing during electronic product manufacturing and electrical performance monitoring during the operation of electrical equipment. However, due to the transient and random nature of flicker, traditional detection methods often have significant limitations.

[0003] In the existing technology, flicker detection methods mainly include the following categories: first, direct detection methods based on photoelectric sensors. Although this type of method has a fast response speed, it is easily interfered by ambient light and difficult to perform regional and localized detection; second, detection methods based on traditional image processing algorithms. This type of method uses fixed thresholds and parameter settings. Although it can achieve basic detection functions under ideal conditions, it is difficult to adapt to interference factors such as lighting changes and perspective offset in actual industrial environments; third, detection methods based on deep learning. Although it has a high recognition accuracy, it has a large amount of calculation and requires a large amount of labeled data for training, which has great limitations in actual industrial applications.

[0004] In particular, detection in high-intensity electromagnetic environments presents unique challenges. Traditional electronic sensors are susceptible to electromagnetic interference, which affects detection reliability; and common visual detection algorithms are often unable to effectively cope with complex environmental interference.

[0005] Therefore, there is an urgent need for a visual inspection method that has strong anti-interference ability, high real-time performance, and is suitable for a variety of industrial scenarios. Summary of the Invention

[0006] The purpose of the present invention is to provide a real-time visual detection method for electric spark flicker based on a regional integration algorithm, which can effectively suppress environmental interference and significantly improve processing efficiency. The method can be applied to scenarios such as quality inspection in industrial production processes, equipment operation status monitoring, and online inspection of automated production lines.

[0007] To achieve the above object, the present invention provides a real-time visual detection method for electric spark flicker based on a regional integration algorithm, comprising the following steps:

[0008] S1, collect video data stream and perform preprocessing and parameter setting;

[0009] S2, based on the YOLO deep learning network, performs real-time target detection and locates the ROI area;

[0010] S3, apply multi-parameter combination optimization technology to perform image enhancement on the ROI area;

[0011] S4, spark feature extraction based on regional brightness integration algorithm;

[0012] S5, using a dual-threaded parallel architecture for data processing and analysis;

[0013] S6. Spark feature recognition is achieved by combining dynamic threshold mechanism;

[0014] S7. Realize real-time output of test results and exception handling.

[0015] Preferably, S1 is specifically:

[0016] S1.1. Industrial camera configuration and initialization, configure the initial parameters of the industrial camera: initial sampling frame rate fps init ;

[0017] S1.2. Collect grayscale image sequence, the calculation formula is:

[0018] I(x,y,t)=0.299R(x,y,t)+0.587G(x,y,t)+0.114B(x,y,t);

[0019] Where (x, y) is the pixel coordinate, t is the time series, R, G, and B represent the pixel values of the red, green, and blue channels respectively;

[0020] S1.3. Extract the foreground target D(x, y, t) based on the background difference method and obtain the spark coordinate sequence P using contour recognition:

[0021] D(x,y,t)=|I(x,y,t)-B(x,y)|;

[0022] P={(x i ,y i )|i=1,2,…,n};

[0023] Among them, B(x,y) is the background model, and n is the ordinal number for sorting the sparks in chronological order;

[0024] S1.4. Calculate the spatial distribution characteristics based on the spark coordinate sequence to achieve adaptive optimization of the frame rate.

[0025] Preferably, in S1.4, the calculation formula of the spatial distribution characteristics is:

[0026]

[0027] Among them, for the spark coordinate pairs (x i ,x i+1 ), if and only if x i+1 >x i When , it constitutes an effective interval, m is the number of effective intervals, and a is the number of preset spark areas;

[0028] The frame rate adaptive optimization is specifically as follows:

[0029] When Δx is satisfied avg >k×Δx theory When , k is the adjustment coefficient, if sparks are missed during shooting due to insufficient frame rate, the sampling frame rate should be increased:

[0030] fps new =min(fps current +Δfps,fps max );

[0031] Among them, fps curerent is the current frame rate of the camera, Δfps is the frame rate increment, fps max is the maximum frame rate of the camera;

[0032] If Δx is not satisfied avg >k×Δx theory , then set this frame rate as the final frame rate and set the final frame rate to the industrial camera.

[0033] Preferably, S2 is specifically:

[0034] S2.1. Construct an optimized deep convolutional neural network structure, including a backbone feature extraction network, a feature pyramid network, and a detection head network. The backbone network adopts an alternating stacked convolutional structure, the feature pyramid network introduces an adaptive spatial feature fusion module, and the detection head network adopts a parallel branching structure.

[0035] S2.2. Optimize network performance through attention mechanism, depthwise separable convolution, residual connection structure and adaptive feature aggregation strategy;

[0036] S2.3. Construct the target detection loss function L, which is calculated as follows:

[0037] L=λ loc L loc +λ cls L cls +λ reg L reg ;

[0038] Among them, L loc is the positioning loss, L cls is the classification loss, Lreg is the regression loss, λ is the corresponding weight coefficient;

[0039] S2.4. Based on the detected target bounding box coordinates (x, y, w, h), perform adaptive region expansion ROI expanded , and its calculation formula is:

[0040] ROI expanded =(x-0.5w,y-0.5h,2w,2h).

[0041] Preferably, S3 is specifically:

[0042] S3.1. Define the parameter space, including the brightness gain coefficient a∈[0.5,2.0], the contrast enhancement factor β∈[1.0,3.0], and the gamma correction parameter γ∈[0.4,1.8];

[0043] S3.2. Build an image enhancement model:

[0044] I enhanced (x,y)=α×[I original (x,y)] γ ×β;

[0045] Among them, I enhanced (x,y) is the enhanced image, I original (x,y) is the original image;

[0046] S3.3, using a multi-threaded parallel computing framework for parameter optimization;

[0047] S3.4. Parameter selection based on the scoring results;

[0048] S3.5. Apply the optimal parameters for image enhancement.

[0049] Preferably, S4 is specifically:

[0050] S4.1. Preprocess the ROI image with Gaussian filtering. smooth (x,y):

[0051] I smooth (x,y)=G σ *I roi (x,y);

[0052] Among them, G σ is a Gaussian kernel with a standard deviation of σ, I roi (x,y) is the original image of the ROI area;

[0053] S4.2. Construct a multi-level brightness integration structure and calculate the ROI global average brightness and the ROI local sub-region average brightness:

[0054] ROI global average brightness:

[0055]

[0056] Among them, G and H are the width and height of the ROI area respectively;

[0057] The ROI area is divided into m×m sub-areas, and the average brightness of the ROI local sub-areas is:

[0058]

[0059] in, is the sub-region size, (x i ,y j ) is the coordinate of the upper left corner of the (i, j)th sub-region;

[0060] S4.3. Luminance feature dynamic gain, calculate the gain coefficient k(i,j):

[0061]

[0062] Where α(t) is the adaptive global gain adjustment coefficient, defined as:

[0063]

[0064] in, is the maximum average brightness value of all sub-regions at time t in the current frame;

[0065] S4.4. Adaptive brightness enhancement of the ROI area:

[0066] I out (x,y)=k(i,j)×I smooth (x,y);

[0067] Among them, I out (x, y) is the enhanced brightness value, I smooth (x, y) is the pre-processed brightness value, (i, j) is the sub-region index where the pixel point (x, y) is located;

[0068] S4.5. Calculate the final average brightness of the area

[0069]

[0070] Preferably, S5 is specifically:

[0071] S5.1. Build a dual-threaded parallel processing architecture based on the producer-consumer model;

[0072] S5.2. Configure the shared buffer size:

[0073] Buffer max =min(0.2×M,2GB);

[0074] Where M is the available physical memory size of the system;

[0075] S5.3, the producer thread performs data collection;

[0076] S5.4, the consumer thread processes the data;

[0077] S5.5. Establish a performance monitoring system to track system resource utilization in real time and adaptively optimize processing performance.

[0078] Preferably, S6 is specifically:

[0079] S6.1. Constructing a sliding window-based brightness sequence acquisition system;

[0080] S6.2. Constructing an adaptive dynamic threshold calculation mechanism:

[0081] Threshild(t)=0.5×min(S large );

[0082] Among them, S large For a large change in sample set;

[0083] S6.3. Establish a spark feature recognition framework based on finite state machine;

[0084] S6.4. Extract spark flickering features;

[0085] S6.5. Generate standardized output results.

[0086] Preferably, S7 is specifically:

[0087] S7.1. Build a real-time display interface;

[0088] S7.2. Detect abnormal situations and define the abnormality type determination function:

[0089]

[0090] Where i is the last flashing position, j is the current flashing position, and d is the number of intervals between the two.

[0091] S7.3, perform data backtracking;

[0092] S7.4. Design the user interaction interface.

[0093] Therefore, the present invention adopts the above-mentioned real-time visual detection method of electric spark flicker based on regional integration algorithm, and the beneficial effects are as follows:

[0094] (1) The present invention integrates the improved YOLO target detection network with the regional brightness integration algorithm, and designs a dual-threaded parallel computing framework based on the producer-consumer model to solve the following problems existing in the prior art: the detection results of traditional electrical detection equipment are unstable due to electromagnetic interference in high-intensity electromagnetic scenes; the detection method based on deep learning has high accuracy but insufficient real-time performance, which makes it difficult to meet the real-time monitoring needs of industrial sites; the detection scheme using the background difference method is easily affected by background interference in complex industrial environments, resulting in a high false alarm rate.

[0095] (2) The present invention effectively suppresses environmental interference and significantly improves processing efficiency. The system processing speed reaches 1200 frames per second and the detection accuracy reaches 99.9%. It can be applied to quality inspection in industrial production processes, equipment operation status monitoring, and online inspection of automated production lines, and has significant engineering application value.

[0096] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0097] Figure 1 This is a schematic diagram of the overall process of an embodiment of a method for real-time visual detection of electric spark flicker based on a regional integration algorithm of the present invention;

[0098] Figure 2 This is a schematic diagram of a dual-thread parallel architecture flow chart of an embodiment of a method for real-time visual detection of electric spark flicker based on a regional integration algorithm of the present invention;

[0099] Figure 3 Schematic diagram of a regional brightness integration algorithm of an embodiment of a method for real-time visual detection of electric spark flicker based on a regional integration algorithm of the present invention;

[0100] Figure 4 This is a schematic diagram of an embodiment of a method for real-time visual detection of electric spark flicker based on a regional integration algorithm of the present invention, which realizes spark feature recognition based on a dynamic threshold mechanism. DETAILED DESCRIPTION

[0101] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0102] Unless otherwise defined, technical and scientific terms used herein shall have the same meanings as those commonly understood by persons of ordinary skill in the art to which this invention pertains. The computer vision processing involved in this invention can be implemented by calling image processing functions provided by OpenCV, while industrial cameras are controlled using a standard camera SDK. Those skilled in the art may adjust these parameters and processing methods based on their understanding of the implementation scheme.

[0103] Example 1

[0104] like Figure 1 As shown, a real-time visual detection method for electric spark flicker based on regional integration algorithm includes the following steps:

[0105] S1. Collect video data stream and perform preprocessing and parameter setting, specifically:

[0106] S1.1. Industrial camera configuration and initialization, configure the initial parameters of the industrial camera:

[0107] fps init =100, where fps init is the initial sampling frame rate.

[0108] S1.2. Collect grayscale image sequence I(x,y,t):

[0109] I(x,y,t)=0.299R(x,y,t)+0.587G(x,y,t)+0.114B(x,y,t)

[0110] Among them, (x, y) is the pixel coordinate, t is the time series, and R, G, and B represent the pixel values of the red, green, and blue channels respectively.

[0111] S1.3. Extract the foreground target D(x, y, t) based on the background difference method and obtain the spark coordinate sequence P using contour recognition:

[0112] D(x,y,t)=|I(x,y,t)·-B(x,y)|;

[0113] P={(x i ,y i )|i=1,2,…,n};

[0114] Among them, B(x,y) is the background model, and n is the ordinal number for sorting sparks in chronological order.

[0115] S1.4. Calculate the spatial distribution characteristics based on the spark coordinate sequence to achieve adaptive optimization of the frame rate.

[0116] Specifically, the calculation formula for spatial distribution characteristics is:

[0117]

[0118] Among them, for the spark coordinate pairs (x i ,x i+1 ), if and only if: x i+1 >x iWhen , it is said to constitute an effective interval, m is the number of effective intervals, and a is the number of preset spark areas.

[0119] Frame rate adaptive optimization, specifically:

[0120] When the condition is met: Δx avg >k×Δx theory When k is the adjustment coefficient, usually k=1.2, it is considered that there are missed sparks during shooting due to insufficient frame rate, so the sampling frame rate is increased:

[0121] fps new =min(fps current +Δfps,fps max );

[0122] Among them, fps current is the current frame rate of the camera, Δfps is the frame rate increment, usually taken as 100, fps max The maximum frame rate of the camera.

[0123] If not satisfied: Δx avg >k×Δx theory , then set this frame rate as the final frame rate.

[0124] Initial parameter setting for industrial cameras: Set the final frame rate to the industrial camera.

[0125] S2: Real-time target detection based on the YOLO deep learning network to locate the ROI (region of interest), specifically:

[0126] S2.1. Construct an optimized deep convolutional neural network structure, including three functional modules: backbone feature extraction network, feature pyramid network and detection head network. The backbone network adopts an alternating stacked convolution structure to enhance feature extraction capability, the feature pyramid network introduces an adaptive spatial feature fusion module to realize multi-scale feature extraction, and the detection head network adopts a parallel branch structure to realize collaborative detection of targets of different scales.

[0127] S2.2. To improve the real-time and accuracy of detection, the network structure is optimized as follows: the attention mechanism is introduced to enhance the feature expression of key areas; depthwise separable convolution is used to reduce computational complexity; a residual connection structure is designed to alleviate the gradient vanishing problem; and an adaptive feature aggregation strategy is used to enhance the multi-scale feature fusion effect.

[0128] S2.3. Construct target detection loss function:

[0129] L=λ loc L loc +λ cls L cls +λ reg Lreg ;

[0130] Among them, L loc is the positioning loss, L cls is the classification loss, L reg is the regression loss, and λ is the corresponding weight coefficient.

[0131] S2.4. ROI extraction strategy: Adaptive region expansion based on the detected target bounding box coordinates (x, y, w, h):

[0132] ROI expanded =(x-0.5w,y-0.5h,2w,2h);

[0133] That is, based on the original detection bounding box, it is expanded in all directions at the same ratio, so that the final ROI area is twice the area of the original detection area, so as to enhance the robustness of the detection results and effectively deal with the target drift problem caused by image jitter.

[0134] S2.5. Design a dynamic threshold screening mechanism to adaptively adjust the confidence threshold according to image quality and environmental conditions, eliminate low-confidence detection results, and output the final ROI area set.

[0135] S3. Apply multi-parameter combination optimization technology to enhance the image in the ROI area, specifically:

[0136] S3.1. Define the parameter space and its search range: brightness gain coefficient a: a∈[0.5,2.0], with a step size of 0.1; contrast enhancement factor β: β∈[1.0,3.0], with a step size of 0.1; gamma correction parameter γ: γ∈[0.4,1.8], with a step size of 0.1.

[0137] S3.2. Build an image enhancement model:

[0138] I enhanced (x,y)=α×[I original (x,y)] γ ×β;

[0139] Among them, I enhanced (x,y) is the enhanced image, I original (x,y) is the original image.

[0140] S3.3. Parameter optimization using a multi-threaded parallel computing framework:

[0141] (1) According to the processor hardware parameters, the parameter space is divided into N subspaces:

[0142] P i ={(α j ,β k,γ l )j∈[1,m],k∈[1,n],l∈[1,p]};

[0143] Where i is the thread number, i∈[1,N], m, n, and p are the number of sampling points of each parameter respectively.

[0144] (2) Each subspace is processed by an independent thread to calculate the contrast ratio:

[0145]

[0146] Among them, Mean(ROI enhanced ) is the average brightness of the ROI area after enhancement, Mean(NonROI enhanced ) is the average brightness of the area outside the ROI area.

[0147] S3.4. Parameter selection based on scoring results:

[0148] (1) Record the maximum contrast ratio:

[0149] Ratio max =max α,β,γ (Ratio(α,β,γ));

[0150] (2) Obtain all parameter combinations that achieve the maximum contrast ratio:

[0151] P candidates ={(α,β,γ)|Ratio(α,β,γ)=Ratio max};

[0152] (3) Select the parameter combination that maximizes the average brightness of the ROI area and makes the image brightest from the candidate parameter group:

[0153]

[0154] S3.5, Apply the optimal parameters for image enhancement: opt ,β opt ,γ opt ) is passed to the industrial camera and set up.

[0155] S4. Spark feature extraction based on regional brightness integration algorithm, specifically:

[0156] S4.1. Perform Gaussian filtering preprocessing on the ROI area image to suppress random noise:

[0157] I smooth (x,y)=G σ *I roi (x,y)

[0158] Among them, G σ is a Gaussian kernel with a standard deviation of σ, I roi (x,y) is the original image of the ROI area.

[0159] S4.2. Construct a multi-level brightness integration structure and calculate the ROI global average brightness and the ROI local sub-region average brightness:

[0160] (1) Calculate the global average brightness of ROI:

[0161]

[0162] Among them, G and H are the width and height of the ROI area respectively;

[0163] (2) Divide the ROI area into m×m sub-regions and calculate the average brightness of the ROI local sub-regions:

[0164]

[0165] in, is the sub-region size, (x i ,y j ) is the coordinate of the upper left corner of the (i, j)th sub-region;

[0166] S4.3. Luminance feature dynamic gain, calculate the gain coefficient k(i,j):

[0167] Calculate the dynamic gain coefficient of each sub-region and establish the matrix: hour:

[0168]

[0169] Otherwise, k(i,j)=1.0.

[0170] Where: α(t) is the adaptive global gain adjustment coefficient, defined as:

[0171]

[0172] in, is the maximum average brightness value of all sub-regions in the current frame at time t. The adaptive global gain adjustment coefficient α(t) is dynamically adjusted according to the image characteristics, with a value range of [1.0, 2.0], which can adaptively adjust the gain intensity.

[0173] S4.4. Adaptive brightness enhancement of the ROI area:

[0174] I out (x,y)=k(i,j)×I smooth (x,y)

[0175] Among them, I out (x, y) is the enhanced brightness value, I smooth (x, y) is the preprocessed brightness value, and (i, j) is the sub-region index where the pixel point (x, y) is located.

[0176] S4.5. Calculate the final average brightness of the area:

[0177]

[0178] in, It is the final average brightness value of the ROI area, which is the key indicator for spark feature judgment.

[0179] S5. Use a dual-threaded parallel architecture for data processing and analysis, specifically:

[0180] S5.1. Build a dual-threaded parallel processing architecture based on the producer-consumer model: the producer thread is responsible for image acquisition and ROI area average brightness calculation; the consumer thread is responsible for data analysis, feature recognition, and result display.

[0181] S5.2. Dynamically configure the shared buffer size based on the system's available memory. Specifically:

[0182] Buffer max =min(0.2×M,2GB)

[0183] Where M is the available physical memory size of the system.

[0184] S5.3. The producer thread collects data. Specific implementation steps:

[0185] First, initialize the circular buffer and dynamically expand and shrink it at runtime based on buffer usage. When the buffer usage exceeds 80% for a sustained period, check the system memory status and expand it to 1.5 times the size if conditions permit. When the buffer usage remains below 30% for a sustained period, shrink it to 0.75 times the current size, but not less than the initial size.

[0186] Secondly, data storage is implemented based on a lock-free queue structure, and a batch write strategy is adopted to improve efficiency;

[0187] Then, implement buffer occupancy monitoring and pause sampling when the occupancy exceeds 95%;

[0188] Finally, a multi-frame parallel processing mechanism based on the number of CPU cores is constructed, and thread local storage is used to ensure data processing security.

[0189] S5.4. The consumer thread processes the data. Specific implementation steps are:

[0190] First, set the minimum data processing batch to implement data prefetching and asynchronous processing;

[0191] Then, a multi-core parallel computing framework is used for data analysis;

[0192] Finally, the processing parameters are dynamically adjusted according to the system load conditions.

[0193] S5.5. Establish a performance monitoring system: Implement thread synchronization based on atomic operations and conditional variables, and design thread status monitoring and fault recovery mechanisms to track system resource utilization in real time and adaptively optimize processing performance.

[0194] S6. Spark feature recognition is achieved by combining the dynamic threshold mechanism, specifically:

[0195] S6.1. Constructing a brightness sequence acquisition mechanism based on a sliding window:

[0196] (1) Set the sliding window parameters: window length W = 100 frames, which is used to capture the complete flickering process and construct the brightness sequence:

[0197]

[0198] Among them, t is the current frame number, is the final average brightness value of the ROI area in the i-th frame;

[0199] (2) Generate inter-frame brightness difference sequence for feature extraction:

[0200]

[0201] S6.2. Constructing an adaptive dynamic threshold calculation mechanism:

[0202] (1) Perform K-means cluster analysis on the absolute value part of the brightness difference sequence and divide the brightness change features into two categories: small changes and large changes:

[0203] C={c1,c2}=kmeans({Δb|Δb>0,Δb∈ΔB(t)},k=2)

[0204] Among them, c1 represents the center of small changes, and c2 represents the center of large changes;

[0205] (2) Extracting a set of samples with significant changes:

[0206] S large ={Δb|||Δb-c2||≤||Δb-c1||,Δb∈ΔB(t)}

[0207] Threshold(t)=0.5×min(S large )

[0208] S6.3. Establish a spark feature recognition framework based on finite state machine:

[0209] (1) Define the brightness status judgment criteria:

[0210]

[0211] (2) Design edge detection function:

[0212]

[0213] S6.4. Extract spark flickering features:

[0214] (1) Maintaining a finite length edge detection sequence:

[0215] E(t)={Edge(i)|i∈[t-K+1,t]}

[0216] Among them, K is the edge detection sequence length, which can be adjusted according to the actual application scenario, and the default value is 10 frames;

[0217] (2) Define the spark flicker judgment criteria: When there is a rising edge and a falling edge in the detection sequence, it is determined to be a valid flicker:

[0218]

[0219] S6.5. Generate standardized output results: Construct output array structure:

[0220] Output(t)=[t,Flash(t),Region(t)]

[0221] Where t is the current frame number; Flash(t) is the flash status flag, 1 indicates that a valid flash is detected, and 0 indicates that no flash is detected; Region(t) is the ROI region number corresponding to when a flash is detected, and is 0 when no flash is detected.

[0222] S7: Realize real-time output of test results and exception handling, specifically:

[0223] S7.1. Build a real-time display interface: First, create a real-time data table based on the output array Output(t) to display the detection results; second, design an abnormal status marking mechanism, using different colors to identify normal and abnormal states.

[0224] S7.2. Detect abnormalities:

[0225] (1) Define the exception type determination function:

[0226]

[0227] Where i is the last flashing position, j is the current flashing position, and d is the number of intervals between the two.

[0228] (2) Function for calculating the number of missed detections:

[0229]

[0230] Where Q is the total number of detection areas, and d is the number of intervals between adjacent flashing positions.

[0231] S7.3. Data Backtracking: Maintain a dynamically sized image cache file stored on disk, which records frame numbers. When an anomaly occurs, locate the frame number where the anomaly occurred, expand forward and backward 30 frames, and outline the problematic ROI with a wireframe. Detailed information is annotated and ultimately output as a video file for users to review.

[0232] S7.4. Design the user interaction interface:

[0233] First, abnormal information is displayed in real time, including the abnormality type, location, and number of missed detections;

[0234] Then, a retrospective video of the abnormal event is generated, and the problem areas are marked;

[0235] Finally, the function of exporting abnormal analysis reports is provided.

[0236] Example 2

[0237] A real-time visual detection system for electric spark flicker for implementing the method of the first embodiment described above includes an industrial camera configuration and system initialization module, a dual-threaded parallel computing framework module, a target detection module, an image enhancement processing module, a regional integration algorithm module, a spark feature recognition mechanism module, and an exception handling and data backtracking module.

[0238] Part 1: Industrial Camera Configuration and System Initialization. Use an industrial camera for image acquisition. First, configure the following parameters through the camera driver: Set the initial camera resolution to 640×480 pixels; Set the initial sampling frame rate to fps. init Set the frame rate to 100fps, disable the auto-exposure function, manually set the exposure time to 2000 microseconds, and set the gain to 0dB. The above parameters can be adjusted adaptively according to the actual application scenario.

[0239] After the industrial camera has completed parameter configuration, the camera SDK is used to acquire images. During the acquisition process, the system automatically converts the acquired color image into a grayscale image according to the following formula:

[0240] I(x,y,t)=0.299R(x,y,t)+0.587G(x,y,t)+0.114B(x,y,t);

[0241] Among them, (x, y) is the pixel coordinate, t is the time series, and R, G, and B represent the pixel values of the red, green, and blue channels respectively.

[0242] Part II: Figure 2 As shown in the figure, a dual-threaded parallel computing framework based on the producer-consumer model consists of an image acquisition thread and a data processing thread. The image acquisition thread acts as a producer, responsible for camera image acquisition and caching; the data processing thread acts as a consumer, responsible for object detection and feature extraction. The two threads exchange data via a circular queue, whose size is dynamically adjusted based on the system's available memory:

[0243] Buffer max =min(0.2×M,2GB);

[0244] Where M is the amount of available physical memory in the system. If the system detects that the queue usage exceeds 80% for a sustained period, it will expand the queue size by 1.5 times, if memory allows. If the usage remains below 30%, it will shrink the queue to 0.75 times its current size, but not less than the initial size.

[0245] Part 3: Object detection based on the improved YOLO network. The OpenCV deep learning module is used to load the optimized YOLO object detection network. The network structure consists of three functional modules: the feature extraction backbone network, the feature pyramid network, and the detection head network. After the detection network outputs the target bounding box coordinates, the system automatically expands the ROI area:

[0246] ROI expanded =(x-0.5w,y-0.5h,2w,2h);

[0247] Where (x, y, w, h) are the parameters of the original detection bounding box. The expanded ROI area is twice the size of the original detection area. This design effectively addresses the issue of target drift caused by image jitter.

[0248] Part 4: Image enhancement processing with multi-parameter combination optimization. Define the parameter space: brightness gain coefficient a∈[0.5,2.0], contrast enhancement factor β∈[1.0,3.0], gamma correction parameter γ∈[0.4,1.8]. Use OpenCV's image processing functions to implement the image enhancement model:

[0249] I enhanced (x,y)=α×[I original (x,y)] γ ×β;

[0250] Among them, I enhanced (x,y) is the enhanced image, I original (x,y) is the original image. This model fully considers the brightness, contrast and nonlinear characteristics of the image and can effectively enhance the feature expression of the ROI area.

[0251] Part V: If Figure 3 As shown in the figure, the implementation process of the regional integration algorithm is as follows. First, the ROI area is preprocessed by Gaussian filtering:

[0252] I smooth (x,y)=G σ *I roi (x,y);

[0253] Among them, G σ is a Gaussian kernel with a standard deviation of σ, I roi (x,y) is the original image of the ROI area. Then a multi-level brightness integration structure is constructed to calculate the global average brightness:

[0254]

[0255] Where W and H are the width and height of the ROI area respectively. Figure 3 As shown, the ROI area is divided into 4×4 sub-areas and the local average brightness is calculated:

[0256]

[0257] in, is the sub-region size, (i, j) is the sub-region index.

[0258] Part VI: Figure 4 As shown in the figure, a spark feature recognition mechanism based on dynamic threshold is adopted. A sliding window method is used to construct a 100-frame brightness sequence. The K-means clustering algorithm of OpenCV is used to classify the brightness change features into two categories: small changes and large changes. The large change sample set is extracted to calculate the dynamic threshold:

[0259] Threshold(t)=0.5×min(S large );

[0260] Among them, S large The sparkle state is determined based on this threshold.

[0261] Part 7: Abnormal Handling and Data Backtracking. This complete abnormal handling mechanism includes the identification of three abnormality types: multiple flickers, forward leakage flickers, and backward leakage flickers. It also implements a hard disk-based data backtracking function, recording 30 frames before and after the abnormality occurs, and marking the problem area with a rectangular frame during playback.

[0262] Therefore, the present invention adopts the above-mentioned real-time visual detection method of electric spark flicker based on regional integration algorithm to effectively suppress environmental interference and significantly improve processing efficiency. It can be applied to quality inspection in industrial production processes, equipment operation status monitoring, and online detection of automated production lines.

[0263] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A real-time visual detection method for electric spark flicker based on regional integration algorithm, characterized in that: The following steps are involved: S1, collect video data stream and perform preprocessing and parameter setting; S2, based on the YOLO deep learning network, performs real-time target detection and locates the ROI area; S3, apply multi-parameter combination optimization technology to perform image enhancement on the ROI area; S4, spark feature extraction based on regional brightness integration algorithm; S5, using a dual-threaded parallel architecture for data processing and analysis; S6. Spark feature recognition is achieved by combining dynamic threshold mechanism; S7. Realize real-time output of test results and exception handling.

2. The method for real-time visual detection of electric spark flicker based on regional integration algorithm according to claim 1, characterized in that: S1 is specifically: S1.

1. Industrial camera configuration and initialization, configure the initial parameters of the industrial camera: initial sampling frame rate fps init ; S1.

2. Collect grayscale image sequence, the calculation formula is: I(x,y,t)=0.299R(x,y,t)+0.587G(x,y,t)+0.114B(x,y,t); Where (x, y) is the pixel coordinate, t is the time series, R, G, and B represent the pixel values of the red, green, and blue channels respectively; S1.

3. Extract the foreground target D(x, y, t) based on the background difference method and obtain the spark coordinate sequence P using contour recognition: D(x,y,t)=|I(x,y,t)-B(x,y)|; P={(x i ,y i )|i=1,2,...,n}; Among them, B(x, y) is the background model, and n is the ordinal number for sorting the sparks in chronological order; S1.

4. Calculate the spatial distribution characteristics based on the spark coordinate sequence to achieve adaptive optimization of the frame rate.

3. The method for real-time visual detection of electric spark flicker based on regional integration algorithm according to claim 2, characterized in that: In S1.4, the calculation formula for the spatial distribution characteristics is: Among them, for the spark coordinate pairs (x i , x i+1 ), if and only if x i+1 >x i When , it constitutes an effective interval, m is the number of effective intervals, and a is the number of preset spark areas; The frame rate adaptive optimization is specifically as follows: When Δx is satisfied avg >k×Δx theory When , k is the adjustment coefficient, if sparks are missed during shooting due to insufficient frame rate, the sampling frame rate should be increased: fps new =min(fps current +Δfps,fps max ); Among them, fps current is the current frame rate of the camera, Δfps is the frame rate increment, fps max is the maximum frame rate of the camera; If Δx is not satisfied avg >k×Δx theory , then set this frame rate as the final frame rate and set the final frame rate to the industrial camera.

4. The method for real-time visual detection of electric spark flicker based on regional integration algorithm according to claim 1, characterized in that: S2 is specifically: S2.

1. Construct an optimized deep convolutional neural network structure, including a backbone feature extraction network, a feature pyramid network, and a detection head network. The backbone network adopts an alternating stacked convolutional structure, the feature pyramid network introduces an adaptive spatial feature fusion module, and the detection head network adopts a parallel branching structure. S2.

2. Optimize network performance through attention mechanism, depthwise separable convolution, residual connection structure and adaptive feature aggregation strategy; S2.

3. Construct the target detection loss function L, which is calculated as follows: L=λ loc L loc +λ cls L cls +λ reg L reg ; Among them, L loc is the positioning loss, L cls is the classification loss, L reg is the regression loss, λ is the corresponding weight coefficient; S2.

4. Based on the detected target bounding box coordinates (x, y, w, h), perform adaptive region expansion ROI expanded , and its calculation formula is: <h2 style=";text-align:left;direction:ltr">ROI<h2 style=";text-align:left;direction:ltr"> expanded <h2 style=";text-align:left;direction:ltr"> (x-0.5w,y-0.5h,2w,2h) 5. The method for real-time visual detection of electric spark flicker based on regional integration algorithm according to claim 1, characterized in that: S3 specifically: S3.

1. Define the parameter space, including the brightness gain coefficient a∈[0.5,2.0], the contrast enhancement factor β∈[1.0,3.0], and the gamma correction parameter γ∈[0.4,1.8]; S3.

2. Build an image enhancement model: I enhanced (x,y)=α×[I original (x,y)] γ ×β; Among them, I enhanced (x, y) is the enhanced image, I original (x, y) is the original image; S3.3, using a multi-threaded parallel computing framework for parameter optimization; S3.

4. Parameter selection based on the scoring results; S3.

5. Apply the optimal parameters for image enhancement.

6. The method for real-time visual detection of electric spark flicker based on regional integration algorithm according to claim 1, characterized in that: S4 is specifically: S4.

1. Preprocess the ROI image with Gaussian filtering. smooth (x, y): I smooth (x,y)=G σ *I roi (x,y); Among them, G σ is a Gaussian kernel with a standard deviation of σ, I roi (x, y) is the original image of the ROI area; S4.

2. Construct a multi-level brightness integration structure and calculate the ROI global average brightness and the ROI local sub-region average brightness: ROI global average brightness: Among them, G and H are the width and height of the ROI area respectively; The ROI area is divided into m×m sub-areas, and the average brightness of the ROI local sub-areas is: in, is the sub-region size, (x i ,y j ) is the coordinate of the upper left corner of the (i, j)th sub-region; S4.

3. Luminance feature dynamic gain, calculate the gain coefficient k(i, j): Where α(t) is the adaptive global gain adjustment coefficient, defined as: in, is the maximum average brightness value of all sub-regions at time t in the current frame; S4.

4. Adaptive brightness enhancement of the ROI area: I out (x,y)=k(i,j)×I smooth (x,y); Among them, I out (x, y) is the enhanced brightness value, I smooth (x, y) is the pre-processed brightness value, (i, j) is the sub-region index where the pixel point (x, y) is located; S4.

5. Calculate the final average brightness of the area 7. The method for real-time visual detection of electric spark flicker based on regional integration algorithm according to claim 1, characterized in that: S5 is specifically: S5.

1. Build a dual-threaded parallel processing architecture based on the producer-consumer model; S5.

2. Configure the shared buffer size: Buffer max =min(0.2×M,2GB); Where M is the available physical memory size of the system; S5.3, the producer thread collects data; S5.4, the consumer thread processes the data; S5.

5. Establish a performance monitoring system to track system resource utilization in real time and adaptively optimize processing performance.

8. The method for real-time visual detection of electric spark flicker based on regional integration algorithm according to claim 1, characterized in that: S6 specifically: S6.

1. Constructing a sliding window-based brightness sequence acquisition system; S6.

2. Constructing an adaptive dynamic threshold calculation mechanism: Threshold(t)=0.5×min(S large ); Among them, S large For a large change in sample set; S6.

3. Establish a spark feature recognition framework based on finite state machine; S6.

4. Extract spark flicker features; S6.

5. Generate standardized output results.

9. The method for real-time visual detection of electric spark flicker based on regional integration algorithm according to claim 1, characterized in that: S7 specifically: S7.

1. Build a real-time display interface; S7.

2. Detect abnormal situations and define the abnormality type determination function: Where i is the last flashing position, j is the current flashing position, and d is the number of intervals between the two. S7.3, perform data backtracking; S7.

4. Design the user interaction interface.

Images