A high-speed visual arc detection method based on pulse neural network

Abstract

This invention belongs to the field of arc detection and brain-like computing technology, and relates to a high-speed visual arc detection method based on spiking neural networks. By combining high-speed image sequence sliding window segmentation with pulse coding, this invention fully utilizes the inherent dynamic characteristics of spiking neural networks in the time dimension, effectively capturing the complete temporal information of the transient arc process and solving the problem of insufficient utilization of temporal features in traditional methods. Utilizing the event-driven computation paradigm of spiking neural networks, neurons only fire pulses when the membrane potential reaches a threshold, significantly reducing computational load and power consumption. It achieves real-time processing of high-frame-rate images without relying on high-performance GPUs, significantly reducing hardware deployment costs and power consumption. Through event-driven computation and sparse pulse transmission, the requirements for storage and bandwidth are significantly reduced, providing a technical solution for high-speed visual arc detection that balances detection accuracy, real-time performance, and deployment economy, with broad application prospects.

Description

Technical Field

[0001] This invention belongs to the field of arc detection and brain-like computing technology, and relates to a high-speed visual arc detection method based on spiking neural networks. Background Technology

[0002] Electric arcs are a common fault phenomenon in electrical systems, characterized by strong light, high temperature, and electromagnetic radiation. If not detected and addressed promptly, they can easily cause fires, equipment damage, and even personal injury. Traditional arc detection methods primarily rely on monitoring electrical parameters, such as analyzing zero-wave characteristics, high-frequency noise, or voltage spikes in current waveforms to identify arcs. However, these methods have inherent limitations: firstly, the response of electrical signals typically lags behind the actual occurrence of the arc, making it difficult to achieve rapid early warning at the millisecond or even microsecond level; secondly, in complex industrial environments, electrical signals are susceptible to electromagnetic interference and load fluctuations, leading to decreased detection accuracy and high false alarm and false negative rates. Furthermore, electrical detection methods cannot provide spatial location information of the arc, hindering rapid fault location and handling.

[0003] With the development of high-speed imaging technology, vision-based arc detection has gradually become a research hotspot. By capturing the transient light intensity changes generated by an arc using a high-speed camera, arc events can be directly observed and identified. Existing visual detection methods mostly use traditional image processing or convolutional neural networks to classify single-frame images. Although these methods can identify arc features to a certain extent, they still have significant shortcomings: First, an arc is a transient process with obvious temporal evolution, typically lasting only a few milliseconds from its generation and development to its extinction. A single-frame image cannot fully represent its dynamic characteristics, and the lack of temporal information limits the detection accuracy. Second, traditional convolutional neural networks have a large number of parameters and high computational complexity. In high frame rate scenarios (such as 5000 frames / second or higher), the hardware cost required for real-time processing is extremely high. This not only requires expensive high-performance GPUs or dedicated acceleration chips but also brings huge power consumption and heat dissipation pressure, making it difficult to deploy on edge devices such as power inspection robots, drones, or embedded protection devices, which seriously restricts the practical application scope of visual arc detection technology.

[0004] As a third-generation neural network, spiking neural networks (SNNs) are inspired by the information processing mechanisms of biological nervous systems, possessing inherent advantages in power consumption and timing modeling. Unlike traditional artificial neural networks that require continuous multiplication and addition operations, SNNs encode information in the time dimension using discrete pulses. Neurons only fire pulses when the accumulated membrane potential reaches a threshold, remaining silent most of the time, naturally exhibiting event-driven and low-power characteristics. This brain-like computing paradigm can achieve online processing of high-frame-rate image sequences with extremely low energy consumption, and the dynamic characteristics of spiking neurons in the time dimension highly match the transient process of an electric arc, capturing timing dependencies without complex loop structures. However, how to effectively encode high-speed image sequences into pulse signals, design SNN structures that can adapt to edge computing devices, and fully leverage their advantages of low power consumption and high timeliness remain challenging issues that urgently need to be addressed in the current technological field.

[0005] This invention addresses the technical bottlenecks of existing visual arc detection methods, such as insufficient temporal modeling capabilities, high computational overhead, and difficulties in edge deployment. It proposes a high-speed arc detection method based on a neuromorphic spiking neural network. This method utilizes the discrete pulse coding mechanism and event-driven characteristics of spiking neural networks to effectively extract transient temporal features of the arc while significantly reducing computational power consumption, providing a feasible solution for real-time and lightweight deployment of arc detection. Summary of the Invention

[0006] To address the technical bottlenecks of existing high-speed visual arc detection methods, such as insufficient utilization of temporal features, high computational overhead, and difficulties in edge deployment, this invention provides a high-speed visual arc detection method based on spiking neural networks. By introducing the event-driven computation paradigm of neuromorphic spiking neural networks and combining sliding window segmentation and pulse coding strategies, a lightweight spiking neural network model is constructed to achieve real-time arc recognition of high frame rate image sequences, effectively solving the technical problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A high-speed visual arc detection method based on a spiking neural network includes the following steps:

[0009] Step 1: Image Acquisition

[0010] Acquire a sequence of images of the scene to be detected, captured continuously by a high-speed camera at a preset frame rate;

[0011] Step 2: Image Processing

[0012] The image sequence of the scene to be detected is divided into several time windows by a preset window length of N frames and a sliding step of M frames. Each time window contains N consecutive frames of images, and there is overlap between adjacent windows. Each frame of the image in each time window is downsampled and normalized to normalize the pixel values ​​to the [0,1] range to form a floating-point pixel representation.

[0013] Step 3: Pulse coding

[0014] Rate encoding is performed on each frame of the image within each time window. A Poisson pulse sequence is generated based on the normalized pixel values. The pulse representations of each frame of the image within the window are combined in chronological order to form a set of spatiotemporal pulse sequences corresponding to that window.

[0015] Step 4: Spiking Neural Network Intelligent Algorithm

[0016] A convolutional spiking neural network is constructed, comprising at least two convolutional layers, spiking neuron layers alternately connected to the convolutional layers, pooling layers, and fully connected classification layers; wherein the spiking neuron layers adopt the LIF (Leaky Integrate-and-Fire) neuron model, and an alternative gradient method is used to achieve backpropagation training;

[0017] Using labeled arc windows as positive samples and normal windows as negative samples, a weighted cross-entropy loss function or oversampling strategy is used to suppress the imbalance between positive and negative samples. The network is trained by a gradient descent optimizer, and the model performance is monitored on the validation set to select the optimal parameters, thus obtaining a network model with arc recognition capability.

[0018] Step 5: Output of test results

[0019] The real-time acquired image sequence is segmented by a sliding window, downsampled, normalized, and pulse-coded before being input into the trained model. The model outputs a classification result to determine whether the current window contains an electric arc event. If multiple consecutive windows are determined to contain an electric arc, an electric arc alarm signal is triggered.

[0020] In the second step of sliding window segmentation, the window length N ranges from 20 to 50 frames, and the sliding step size M ranges from N / 3 to N / 2, so that the time length corresponding to each window is between 2 milliseconds and 5 milliseconds, which matches the typical duration of the electric arc.

[0021] The third step of pulse coding is specifically operated as follows: For the i-th frame image within each time window, a corresponding pulse matrix is ​​generated based on the normalized pixel value matrix; for each pixel position, a Poisson pulse sequence of length T is generated using its normalized pixel value p as the pulse firing probability, where the probability of generating a pulse at each time step is equal to p; the pulse sequences of all pixel positions are used in parallel to form the pulse representation of the frame image; the pulse representations of each frame image within the window are combined in chronological order to form the spatiotemporal pulse sequence set corresponding to the window.

[0022] In the fourth step of the intelligent spiking neural network algorithm, the convolutional spiking neural network includes: an input layer for receiving the spatiotemporal pulse sequence after pulse encoding; a first convolutional layer with a kernel size of 3×3, a stride of 1, padding of 1, and 32 output channels, followed by a first LIF neuron layer; a second convolutional layer with a kernel size of 3×3, a stride of 1, padding of 1, and 64 output channels, followed by a second LIF neuron layer; a pooling layer using max pooling with a kernel size of 2×2 and a stride of 2; a fully connected layer that flattens the pooled feature map before inputting it into the fully connected layer, with an output dimension of 128, followed by a third LIF neuron layer; and an output layer that maps the 128-dimensional features to a 2-dimensional output, corresponding to the two categories of "existing arc" and "non-existent arc".

[0023] In the fourth step of the intelligent spiking neural network algorithm, during training, the number of positive and negative samples in the training set is calculated, and a higher loss weight is assigned to the positive samples. The positive sample weight is set as the ratio of the number of negative samples to the number of positive samples. A gradient descent optimizer is used, an initial learning rate is set and a learning rate decay strategy is adopted, the model performance is monitored on the validation set, and the optimal model parameters are selected.

[0024] The beneficial effects of this invention are:

[0025] This invention combines high-speed image sequence sliding window segmentation with pulse coding, fully utilizing the inherent dynamic characteristics of spiking neural networks (SNNs) in the time dimension. This effectively captures the complete temporal information of the transient process of an electric arc, solving the problem of insufficient utilization of temporal features in traditional methods. Utilizing the event-driven computation paradigm of SNNs, neurons fire pulses only when the membrane potential reaches a threshold, significantly reducing computational load and power consumption. It achieves real-time processing of high-frame-rate images without relying on high-performance GPUs, significantly reducing hardware deployment costs and power consumption, enabling visual arc detection technology to be deployed in edge devices such as drones and inspection robots. Simultaneously, a weighted loss function effectively suppresses the impact of positive-negative sample imbalance on model training, improving the model's robustness in practical applications. The lightweight SNN architecture provided by this invention, through event-driven computation and sparse pulse transmission, significantly reduces storage and bandwidth requirements, offering a technical solution for high-speed visual arc detection that balances detection accuracy, real-time performance, and deployment economy, with broad application prospects. Attached Figure Description

[0026] Figure 1 This is an overall flowchart of the method of the present invention.

[0027] Figure 2 This is a schematic diagram of the sliding window segmentation.

[0028] Figure 3(a) is a pulse coding example of normalized gray values ​​for a 2×2 pixel matrix.

[0029] Figure 3(b) is a schematic diagram of pulse coding for the pulse sequence generated based on rate coding for each pixel, with vertical lines representing pulses.

[0030] Figure 4 This is a diagram of a spiking neural network structure. Detailed Implementation

[0032] Please see Figures 1 to 4 This invention provides a high-speed visual arc detection method based on a spiking neural network. This embodiment uses a real-world arc detection scenario as an example to explain the implementation process of this invention in detail.

[0033] Figure 1 This is an overall flowchart of the method of the present invention, showing the core steps from image acquisition to arc output. Figure 2 Figure 3 is a schematic diagram of sliding window segmentation, showing the window length N, step size M, and overlap relationship. Figure 3 is a schematic diagram of pulse coding, showing the process of converting pixel grayscale values ​​into pulse sequences in a single frame image. Specifically, Figure 3(a) is a schematic diagram of pulse coding for an example of normalized grayscale values ​​of a 2×2 pixel matrix; Figure 3(b) is a schematic diagram of pulse coding for pulse sequences generated based on rate coding for each corresponding pixel, with vertical lines representing pulses. Figure 4 This is a diagram of a spiking neural network, showing the connections between convolutional layers, LIF neuron layers, pooling layers, and fully connected layers.

[0034] A high-speed visual arc detection method based on a spiking neural network, comprising the following steps:

[0035] Step 1: Image Acquisition

[0036] Reference Figure 1 The "Image Acquisition Unit" in this section continuously captures images of the scene to be detected using a high-speed camera at a preset frame rate, obtaining a raw image sequence. For example, a MEMRECAM high-speed camera is used to capture images at 10,000 frames per second, with an image resolution of 896×1280 pixels, and each frame is labeled with an acquisition timestamp. This step provides raw data for subsequent processing.

[0037] Step 2: Sliding window segmentation and preprocessing

[0038] like Figure 2 As shown, the image sequence is segmented using a window length of N frames and a sliding step size of M frames to generate several time windows. Each window contains N consecutive frames of images, with overlap between adjacent windows (the number of overlapping frames is NM). The window length N can be adjusted according to actual needs, and the sliding step size M ranges from N / 3 to N / 2, ensuring that the time length corresponding to each window is between 2 milliseconds and 5 milliseconds, matching the typical duration of an electric arc. In this embodiment, N=30, M=15, and the window duration is 3 milliseconds.

[0039] Then, each frame of the image within each window is downsampled (e.g., by bilinear interpolation) to reduce the resolution to a preset spatial size, denoted as H×W (e.g., H=224, W=224 pixels), to reduce subsequent computation. Finally, the downsampled pixel values ​​are normalized to the [0,1] interval to form a floating-point pixel representation.

[0040] Step 3: Pulse coding

[0041] As shown in Figure 3, rate coding (Poisson coding) is performed on each frame of the image within each time window. Specifically, for each pixel position, its normalized gray value p is used as the pulse firing probability to generate a pulse sequence of length T (T is a preset time step; in this embodiment, T=10). Each time step independently generates a pulse with probability p (1 indicates a pulse, 0 indicates no pulse). The pulses of all pixels at the same time step constitute a pulse matrix. T time steps yield T pulse matrices, which are arranged in chronological order to form the pulse representation of the frame image. The pulse representations of all frames within the window are sequentially concatenated to obtain the spatiotemporal pulse sequence corresponding to that window, with a total time step of N×T.

[0042] Step 4: Spiking Neural Network Inference

[0043] like Figure 4 As shown, this step will construct a convolutional spiking neural network, including an input layer, at least two convolutional layers (each followed by a LIF neuron layer), a pooling layer, a flattening layer, a fully connected layer (followed by a LIF neuron layer), and an output layer (for binary classification). The membrane potential dynamics of the LIF neurons follow a leakage integral-fire mechanism, and a substitution gradient method is used to achieve backpropagation training.

[0044] The spatiotemporal pulse sequence obtained in step 3 is input into the network. At each time step, only the pulse position is multiplied and added (event-driven). The LIF neuron updates the membrane potential and decides whether to fire a pulse. After all time steps, the cumulative membrane potential or pulse count of the two neurons in the output layer is read to obtain the original scores for the two categories, denoted as s0 (no arc) and s1 (with arc), respectively. These two scores are converted into probability distributions using the Softmax function, as shown in Equation 1. After Softmax normalization, the arc probability Parc is obtained. If Parc > 0.5, it is determined that an arc exists in the current window.

[0045] (1)

[0046] Step 5: Model Training

[0047] This step is optional. This invention uses labeled arc windows (positive samples) and normal windows (negative samples) to train a spiking neural network. To address the high cost and scarcity of arc samples, one or more of the following training strategies can be used:

[0048] • Weighted Loss: The weighted cross-entropy loss function is used, assigning higher loss weights to positive samples. The weights can be set as the ratio of the number of negative samples to the number of positive samples, or determined empirically (e.g., positive sample weight = 10, negative sample weight = 1).

[0049] • Oversampling: Positive samples are repeatedly sampled during data loading to balance the number of positive and negative samples in each training batch. The oversampling factor is usually set to the integer part of the number of negative samples divided by the number of positive samples.

[0050] • Undersampling: Randomly discarding some negative samples to make the number of positive and negative samples close. However, considering that negative samples contain rich normal patterns, undersampling may lose information and is generally not the first choice.

[0051] • Data augmentation: Randomly translate, rotate, scale, or add noise to the positive sample window to generate more variations and improve the model's generalization ability. Augmented positive samples can effectively expand small sample datasets.

[0052] • Transfer learning: Pre-training a spiking neural network on a large-scale dataset of similar tasks (such as other transient spot detection) and then fine-tuning it on the local dataset can significantly improve the learning effect of small samples.

[0053] • Synthetic minority class oversampling technique: interpolating positive samples in the feature space to generate new samples, while ensuring the rationality of the temporal structure.

[0054] In this embodiment, a strategy combining weighted loss (positive sample weight = number of negative samples / number of positive samples) with random oversampling is adopted. The Adam optimizer is used to train the network with an initial learning rate of 1e-3. The F1 score is monitored on the validation set to select the optimal model parameters.

[0055] Step 6: Arc alarm output

[0056] At this step, the classification results for each window will be received in real time. Since the transient duration of an electric arc typically spans multiple time windows (for example, in this embodiment, the window duration is 3 milliseconds, while the typical arc duration is approximately 2-5 milliseconds, therefore, an arc often appears consecutively in 2-3 adjacent windows), a continuous window determination strategy is adopted to avoid false alarms caused by random noise or transient interference (such as light flickering) in a single window. Specifically, if the current window is determined to have an "electric arc," it is recorded; if multiple consecutive windows (e.g., 3 consecutive windows) are all determined to have an electric arc, the electric arc event is considered to have actually occurred, and an alarm signal is immediately output.

[0057] The parameters in the above steps (window length N, step size M, pulse coding time step T, number of continuous alarm windows K, etc.) can all be adjusted according to the actual frame rate and arc characteristics. For example, if the frame rate is higher, N can be appropriately increased to maintain the window duration; if the arc duration is shorter, N can be decreased. Any appropriate values ​​selected by those skilled in the art within the scope of this invention fall within the protection scope of this invention.

[0058] It should be noted that, Figures 1 to 4 The accompanying drawings are merely exemplary of the present invention. In actual implementation, the network structure (such as the number of convolutional layers, the number of channels, LIF neuron parameters, etc.) can be adjusted as needed. As long as it does not deviate from the core idea of ​​combining data processing, pulse coding and spiking neural network inference, it falls within the protection scope of the present invention.

[0059] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0060] The above description is merely an example of a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A high-speed visual arc detection method based on a spiking neural network, characterized in that, Includes the following steps: Step 1: Image Acquisition Acquire a sequence of images of the scene to be detected, captured continuously by a high-speed camera at a preset frame rate; Step 2: Image Processing The image sequence of the scene to be detected is divided into several time windows by a preset window length of N frames and a sliding step of M frames. Each time window contains N consecutive frames of images, and there is overlap between adjacent windows. For each frame of the image within each time window, downsampling and normalization are performed to normalize the pixel values ​​to the [0,1] range, forming a floating-point pixel representation; Step 3: Pulse coding Rate encoding is performed on each frame of the image within each time window. A Poisson pulse sequence is generated based on the normalized pixel values. The pulse representations of each frame of the image within the window are combined in chronological order to form a set of spatiotemporal pulse sequences corresponding to that window. Step 4: Spiking Neural Network Intelligent Algorithm A convolutional spiking neural network is constructed, comprising at least two convolutional layers, spiking neuron layers alternately connected to the convolutional layers, pooling layers, and a fully connected classification layer; wherein the spiking neuron layers adopt the Leaky Integrate-and-Fire neuron model, and backpropagation training is implemented using an alternative gradient method; Using labeled arc windows as positive samples and normal windows as negative samples, a weighted cross-entropy loss function or oversampling strategy is used to suppress the imbalance between positive and negative samples. The network is trained by a gradient descent optimizer, and the model performance is monitored on the validation set to select the optimal parameters, thus obtaining a network model with arc recognition capability. Step 5: Output of test results The real-time acquired image sequence is segmented by a sliding window, downsampled, normalized, and pulse-coded before being input into the trained model. The model outputs a classification result to determine whether the current window contains an electric arc event. If multiple consecutive windows are determined to contain an electric arc, an electric arc alarm signal is triggered.

2. The high-speed visual arc detection method based on a spiking neural network as described in claim 1, characterized in that, In the second step of sliding window segmentation, the window length N ranges from 20 to 50 frames, and the sliding step size M ranges from N / 3 to N / 2, so that the time length corresponding to each window is between 2 milliseconds and 5 milliseconds.

3. The high-speed visual arc detection method based on a spiking neural network as described in claim 1, characterized in that, The third step of pulse coding is specifically operated as follows: For the i-th frame image within each time window, a corresponding pulse matrix is ​​generated based on the normalized pixel value matrix; for each pixel position, a Poisson pulse sequence of length T is generated using its normalized pixel value p as the pulse firing probability, wherein the probability of generating a pulse at each time step is equal to p. The pulse sequences at all pixel locations are used in parallel to form the pulse representation of the frame image; the pulse representations of each frame image within the window are combined in chronological order to form the spatiotemporal pulse sequence set corresponding to the window.

4. The high-speed visual arc detection method based on a spiking neural network as described in claim 1, characterized in that, In the fourth step of the intelligent spiking neural network algorithm, the convolutional spiking neural network includes: an input layer for receiving the spatiotemporal pulse sequence after pulse encoding; a first convolutional layer with a kernel size of 3×3, a stride of 1, padding of 1, and 32 output channels, followed by a first LIF neuron layer; a second convolutional layer with a kernel size of 3×3, a stride of 1, padding of 1, and 64 output channels, followed by a second LIF neuron layer; a pooling layer using max pooling with a kernel size of 2×2 and a stride of 2; a fully connected layer that flattens the pooled feature map before inputting it into the fully connected layer, with an output dimension of 128, followed by a third LIF neuron layer; and an output layer that maps the 128-dimensional features to a 2-dimensional output, corresponding to the two categories of "existing arc" and "non-existent arc".

5. The high-speed visual arc detection method based on a spiking neural network as described in claim 1, characterized in that, In the fourth step of the intelligent spiking neural network algorithm, during training, the number of positive and negative samples in the training set is calculated, and a higher loss weight is assigned to the positive samples. The positive sample weight is set as the ratio of the number of negative samples to the number of positive samples. A gradient descent optimizer is used, an initial learning rate is set and a learning rate decay strategy is adopted, the model performance is monitored on the validation set, and the optimal model parameters are selected.

Images