Multi-frame infrared arc detection method and device based on double-current network, terminal equipment and storage medium
By using a multi-frame infrared arc detection method based on a dual-stream network, spatiotemporal features are fused and multi-scale fusion is performed. This solves the problems of traditional arc detection methods being susceptible to environmental interference and the inability of single-frame deep learning to model temporal motion patterns, and achieves high-precision arc fault early warning.
Patent Information
- Application Number
- CN202511051569.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-29
- Publication Date
- 2025-12-12
AI Technical Summary
In existing technologies, traditional arc detection methods are easily affected by environmental interference, and single-frame deep learning methods cannot effectively model temporal motion patterns, leading to problems such as false alarms and missed detections.
A multi-frame infrared arc detection method based on a dual-stream network is adopted. By fusing spatiotemporal features, motion and appearance features are extracted from the arc infrared image using a dual-stream network model. The appearance features are enhanced by motion guidance. Combined with a multi-scale progressive fusion strategy, high-precision segmentation of the arc target is achieved.
It improves the robustness and real-time performance of arc detection in complex industrial scenarios, reduces false alarm rate and false negative rate, and improves detection accuracy.
Smart Images
Figure CN121120495A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of arc detection technology, specifically relating to a multi-frame infrared arc detection method, device, terminal equipment, and storage medium based on a dual-stream network. Background Technology
[0002] In the fields of power systems, industrial equipment, and new energy, electric arc faults are a significant hidden danger causing fires, equipment damage, and even explosions. Infrared arc detection technology, due to its non-contact nature, high sensitivity, and adaptability to complex environments, has become one of the core methods for fault monitoring. However, electric arcs themselves are transient, morphologically variable, and exhibit random motion, posing numerous challenges to traditional detection methods in practical applications. Early arc detection methods primarily relied on threshold segmentation, morphological processing, and manual feature extraction. However, the grayscale difference between the arc and the background in infrared images is significantly affected by factors such as ambient temperature and smoke obstruction, making thresholding methods susceptible to interference and resulting in high false detection rates. Traditional optical flow algorithms have high computational complexity and are sensitive to noise, making them unsuitable for real-time monitoring requirements.
[0003] With the development of deep learning, single-frame arc detection methods based on convolutional neural networks have significantly improved detection accuracy through end-to-end learning. However, these methods rely solely on single-frame spatial features and cannot effectively model the temporal motion patterns of the arc, still exhibiting issues such as false alarms and missed detections in complex scenes. To fuse spatiotemporal information, dual-stream networks have been introduced into the field of video analytics. These networks extract single-frame appearance features through spatial flow and capture inter-frame motion features through temporal flow. However, existing dual-stream network architectures are not adequately optimized for infrared characteristics, making it difficult to efficiently fuse spatiotemporal features from multiple frames.
[0004] Therefore, it is urgent to design a lightweight optical flow estimation module, an adaptive feature fusion strategy, and a long-term modeling mechanism to achieve high-precision, low-latency arc fault early warning in complex industrial scenarios. Summary of the Invention
[0005] To address the problems in existing technologies, such as the susceptibility of traditional arc detection methods to environmental interference and the inability of single-frame deep learning methods to effectively model temporal motion patterns leading to false alarms and missed detections, this invention provides a multi-frame infrared arc detection method based on a dual-stream network. This method improves detection accuracy by fusing spatiotemporal features and also provides corresponding devices, terminal equipment, and storage media.
[0006] The first aspect of this invention provides a multi-frame infrared arc detection method based on a dual-stream network, the method comprising at least the following steps:
[0007] Multiple consecutive arc infrared images are acquired at preset millisecond time intervals, and each infrared image is sequentially input into a pre-trained dual-stream network model.
[0008] The two-stream network model extracts the motion and appearance features of the electric arc from the current frame of the electric arc infrared image, enhances the appearance features through motion-guided enhancement, and performs multi-scale progressive fusion processing on the motion features and the enhanced appearance features.
[0009] Based on the differences in resolution and semantic information of features at different stages of the dual-stream network model, high-level semantics are fused into high-resolution features, and the target segmentation image of the current frame corresponding to the current frame arc infrared image is obtained.
[0010] The current frame target segmentation image is converted into a binary image, and the connected regions of the binary image are marked. The segmentation image is then corrected based on the feature information after fusion processing to obtain the final current frame target segmentation image.
[0011] Valid arc faults are determined based on the final target segmentation image of the current frame.
[0012] In one possible implementation, before acquiring multiple consecutive arc infrared images at preset millisecond time intervals and sequentially inputting each infrared image into a pre-trained dual-stream network model, the method further includes:
[0013] Create a two-stream network model;
[0014] The equipment setup was verified and clear arc images were captured through preliminary experiments.
[0015] Preprocessing of the infrared image training set yields preprocessed training images;
[0016] Obtain training target segmentation image samples corresponding to each preprocessed training image;
[0017] Feature data of the preprocessed training images are extracted using the dual-stream network model, and the model is trained using the segmented image samples of the training target as the output reference until the parameters converge.
[0018] In one possible implementation, the architecture of the two-stream network model is constructed by enhancing the Swin-Unet deep learning segmentation network through the introduction of a parallel two-stream architecture; wherein,
[0019] The architecture of the dual-stream network model includes a transformer architecture and parallel sub-networks with U-shaped feature extraction, thereby promoting feature fusion and utilizing a cross-attention mechanism to continuously exchange modules between modalities. Within the architecture of the dual-stream network model, the cross-transformer module uses deformable sampling to calculate the structural relationship between the two modalities, thereby reshaping the structural information of one modality to align with the corresponding structures of the two modalities within the same local window of the Swing Transformer. Through the interaction of the two modalities, while maintaining the original feature distribution of the modalities, it can enhance relevant segmentation features and reduce the influence of irrelevant features.
[0020] In one possible implementation, extracting the motion and appearance features of the electric arc from the current frame of the arc infrared image using the dual-stream network model includes:
[0021] Dynamic features, i.e. motion features, between multiple frames can be extracted using optical flow networks or 3D convolution, including the displacement vector of the electric arc, flicker frequency, and rate of morphological change.
[0022] Static features, or appearance features, of a single frame of infrared image are extracted using a convolutional neural network, including the shape, temperature gradient, and texture of high-temperature areas.
[0023] In one possible implementation, appearance features are enhanced through motion-guided enhancement, including:
[0024] An attention mask is generated using the motion features of the time branch, including a saliency map of the dynamic region;
[0025] The appearance features are weighted by a multiplicative attention mechanism to suppress static interference by weighting regions related to electric arc motion. Furthermore, based on a cross-attention mechanism, the similarity between the motion features and surrounding window features is calculated using semantic information from the motion information, thereby guiding the appearance features to learn semantics.
[0026] In one possible implementation, the multi-scale progressive fusion processing of motion features and enhanced appearance features includes:
[0027] Shallow high-resolution features and deep low-resolution features are fused in layers through skip connections and upsampling to generate a composite feature map that includes detailed localization and global semantics.
[0028] In one possible implementation, converting the current frame target segmentation image into a binary image and marking the connected regions of the binary image includes:
[0029] Obtain the pixels in the target segmentation image of the current frame and compare their radiance intensity with a preset intensity threshold;
[0030] If the radiation intensity is less than the threshold, the pixel gray value is set to 0; if it is greater than the threshold, it is set to 255, and a binary image is generated.
[0031] Connectivity analysis is used to number the independent regions in the binary image and extract the geometric attributes of each region, including contour, area, and centroid.
[0032] A second aspect of the present invention provides a multi-frame infrared arc detection device based on a dual-stream network, comprising:
[0033] The image acquisition unit is used to acquire multiple consecutive arc infrared images at preset millisecond time intervals, and input each infrared image sequentially into a pre-trained dual-stream network model.
[0034] The feature processing unit is used to extract the motion features and appearance features of the electric arc from the current frame of the electric arc infrared image through the dual-stream network model, enhance the appearance features through motion-guided enhancement, and perform multi-scale progressive fusion processing on the motion features and the enhanced appearance features.
[0035] The segmentation optimization unit is used to fuse high-level semantics into high-resolution features based on the differences in resolution and semantic information of features at different stages of the dual-stream network model, and to obtain the target segmentation image of the current frame corresponding to the current frame arc infrared image.
[0036] The image correction unit is used to convert the current frame target segmentation image into a binary image, mark the connected regions of the binary image, and correct the segmentation image according to the feature information after fusion processing to obtain the final current frame target segmentation image.
[0037] The output unit is used to determine valid arc faults based on the target segmentation image of the current frame.
[0038] A third aspect of the present invention provides a terminal device, including an interactive device, an input / output (I / O) interface, a processor, and a memory, wherein the memory stores program instructions;
[0039] The interactive device is used to acquire user input commands;
[0040] The processor is used to execute program instructions stored in the memory to perform the multi-frame infrared arc detection method based on a dual-stream network as described above.
[0041] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the multi-frame infrared arc detection method based on a dual-stream network as described above.
[0042] Compared with existing technologies, the embodiments of the present invention fuse multi-frame spatiotemporal features through a dual-stream network, converting target detection into image segmentation. Combined with motion guidance and multi-scale fusion strategies, it effectively overcomes the false alarm and missed detection problems of single-frame detection, and improves the robustness and real-time performance of arc detection in complex industrial scenarios.
[0043] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.
[0045] Figure 1 This is a flowchart illustrating the multi-frame infrared arc detection method in the embodiments of this application;
[0046] Figure 2 This is a flowchart illustrating the dual-stream network in an embodiment of this application.
[0047] Figure 3 This is a model structure diagram of the U-shaped parallel feature network in the embodiments of this application;
[0048] Figure 4 This is a model structure diagram of the cross converter module in the embodiments of this application;
[0049] Figure 5 This is a diagram of the deformable convolution model structure in the embodiments of this application;
[0050] Figure 6 This is a schematic diagram of the structure of the multi-frame infrared arc detection device in the embodiments of this application;
[0051] Figure 7 A schematic diagram of a terminal device for multi-frame infrared arc detection in this application embodiment. Detailed Implementation
[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.
[0053] In the specification, claims, and accompanying drawings of this application, the use of terms such as "first" and "second" is for descriptive purposes only, to distinguish different objects, and not to describe a specific order, nor should it be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated.
[0054] The terms “comprising” and “having”, and any variations thereof, used in this document are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps, units, or modules is not limited to the steps, units, or modules listed, but may optionally include steps, units, or modules not listed, or may optionally include other steps, units, or modules inherent to such process, method, product, or device.
[0055] The use of "and / or" or "and / or" in the text implies three parallel options. For example, "A and / or B" could mean option A, option B, or a combination of both A and B.
[0056] The term "embodiment" in this document means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.
[0057] like Figure 1-2 As shown, this embodiment of the invention provides a multi-frame infrared arc detection method based on a dual-stream network.
[0058] S100: Acquire multiple consecutive arc infrared images at preset millisecond time intervals, and input each infrared image sequentially into a pre-trained dual-stream network model.
[0059] For example, time-series sampling continuously acquires 5 frames of infrared image sequence I at 10ms intervals (100Hz). t ,I t+1 ,...,I t+4 Synchronous recording of timestamps ensures inter-frame alignment; preprocessing removes random noise through median filtering and uses bilinear interpolation to uniformly scale the image to 512×512, and inputs the two-stream network with the leading edge time axis stacked as a 5×1×512×512 tensor.
[0060] In nature, all objects can radiate infrared radiation. Therefore, by using a detector to measure the difference in infrared radiation between the target and the background, infrared images formed by different thermal infrared radiation can be obtained.
[0061] In practical applications, arc detection requires high-sensitivity and high-resolution infrared images. To achieve this, we use a high-sensitivity infrared camera to acquire multiple consecutive frames of arc infrared images at preset millisecond intervals (e.g., 10 milliseconds). These images will serve as input data for subsequent processing. To ensure image continuity and clarity, we conduct several pre-experiments before formal data acquisition to verify the correctness of the equipment settings and its ability to capture clear arc images. The results of these pre-experiments will be used to adjust camera parameters, such as exposure time and gain, to obtain optimal image quality.
[0062] It is understood that the two-stream network model is the core of this embodiment, used to extract and fuse the motion and appearance features of the electric arc. In one possible implementation, before acquiring multiple consecutive frames of electric arc infrared images at preset millisecond time intervals and sequentially inputting each infrared image into the pre-trained two-stream network model, the following steps are also included:
[0063] Create a two-stream network model;
[0064] The equipment setup was verified and clear arc images were captured through preliminary experiments.
[0065] Preprocessing of the infrared image training set yields preprocessed training images;
[0066] Obtain training target segmentation image samples corresponding to each preprocessed training image;
[0067] Feature data of the preprocessed training images are extracted using the dual-stream network model, and the model is trained using the segmented image samples of the training target as the output reference until the parameters converge.
[0068] In one possible implementation, such as Figure 3-5 As shown, the architecture of the dual-stream network model is constructed by enhancing the Swin-Unet deep learning segmentation network through the introduction of a parallel dual-stream architecture; wherein,
[0069] The architecture of the two-stream network model includes a transformer architecture and parallel subnetworks with U-shaped feature extraction, thereby promoting feature fusion and utilizing a cross-attention mechanism to continuously exchange modules between modalities. Within the architecture of the two-stream network model, the cross-transformer module uses deformable sampling to calculate the structural relationship between the two modalities, thereby reshaping the structural information of one modality to align with the corresponding structures of the two modalities within the same local window of the SwinTransformer. Through the interaction of the two modalities, while maintaining the original feature distribution of the modalities, relevant segmentation features can be enhanced and the influence of irrelevant features can be reduced.
[0070] Specifically, before using a two-stream network model, we need to train it. The training process includes the following steps:
[0071] Data preprocessing: We preprocess multiple training sets to obtain multiple preprocessed training images. Preprocessing steps include image enhancement, normalization, and other operations to improve the model's generalization ability.
[0072] Obtaining training target segmentation image samples: For each preprocessed training image, we obtain its corresponding target segmentation image samples. These samples will serve as output references during the training process.
[0073] Model Training: Training feature data was extracted from each preprocessed training image using the arc detection network model. The segmented image samples of each training target were used as output references to train the arc detection network model until the model parameters converged. During training, the cross-entropy loss function and the Adam optimizer were used, with a learning rate of 0.0001 and a batch size of 16.
[0074] S200. Extract the motion and appearance features of the electric arc from the current frame of the electric arc infrared image through the dual-stream network model, enhance the appearance features through motion-guided enhancement, and perform multi-scale progressive fusion processing on the motion features and the enhanced appearance features.
[0075] In one possible implementation, extracting the motion and appearance features of the electric arc from the current frame of the arc infrared image using the dual-stream network model includes:
[0076] Dynamic features, i.e. motion features, between multiple frames are extracted using optical flow networks or 3D convolution, including the displacement vector of the electric arc, flicker frequency, and rate of morphological change; static features, i.e. appearance features, of a single frame of infrared image are extracted using convolutional neural networks, including the shape of the high-temperature region, temperature gradient, and texture.
[0077] For example, the temporal branch calculates the optical flow field of adjacent frames through 3D convolution (3×3×3 kernel size, 1×2×2 stride) to extract the displacement vector and flicker frequency of the electric arc (such as 50Hz power frequency features); the spatial branch extracts multi-scale feature maps of 256×256 (details) to 32×32 (semantics) in layers C2-C5 of the encoder through hierarchical feature extraction.
[0078] In one possible implementation, appearance features are enhanced through motion-guided enhancement, including:
[0079] An attention mask is generated using the motion features of the time branch, including a saliency map of the dynamic region; the regions related to the arc motion in the appearance features are weighted by a multiplicative attention mechanism to suppress static interference; and based on a cross-attention mechanism, the similarity between the motion features and the surrounding window features is calculated using the semantic information in the motion information to guide the appearance features to learn semantics.
[0080] In this embodiment, multi-stage features with different resolutions and semantic information are extracted from the two-stream network:
[0081] Low-resolution features: Deep semantic features, which contain rich semantic information but have low resolution;
[0082] High-resolution features: shallow semantic features, with high resolution but less semantic information;
[0083] Starting with low-resolution features, gradually fuse them with high-resolution features; upsample the low-resolution features to the same resolution as the high-resolution features.
[0084] The upsampled low-resolution features are concatenated with the high-resolution features along the channel dimension.
[0085] The spliced features are processed using a two-layer residual structure to adjust the number of channels and further integrate semantic information.
[0086] Specifically, based on the cross-attention mechanism, semantic information can be extracted from motion information through cross-attention, and the similarity between motion features and surrounding window features can be calculated to guide appearance features to learn semantics. The main goal of the cross-attention mechanism is to establish correlation between two modalities using attention mechanisms, which is achieved by calculating the correlation between feature A and feature B. The encoding vector of feature B is used as the query and value, while the encoding vector of feature A is used as the key. The query operation maps the correlation between each point in feature B and the corresponding point in feature A through correlation calculation with the key.
[0087] The formula for calculating cross-attention is as follows:
[0088]
[0089] Q b K a and V b Q represents the query, key, and value matrix, respectively. b and V b Encoding from FeatureB, Ka Encoding from FeatureA. d represents the feature dimension of the key.
[0090] In one possible implementation, the multi-scale progressive fusion processing of motion features and enhanced appearance features includes:
[0091] Shallow high-resolution features and deep low-resolution features are fused in layers through skip connections and upsampling to generate a composite feature map that includes detailed localization and global semantics.
[0092] For example, the enhanced appearance features and motion features are fused according to the following steps: the deep semantic features (32×32) are upsampled to the same resolution as the shallow features (256×256); the deep and shallow features are spliced together by skip connections, and a composite feature map is generated by 3×3 convolution; the above process is repeated until a fused feature map of 512×512 is generated.
[0093] S300. Based on the differences in resolution and semantic information of the features at different stages of the dual-stream network model, high-level semantics are fused into high-resolution features, and the target segmentation image of the current frame corresponding to the current frame arc infrared image is obtained.
[0094] For example, the fused feature map is activated by 1×1 convolution + Sigmoid, and the output is a pixel-level arc probability map. Areas with a threshold of 0.7 or higher are identified as arc regions.
[0095] S400. Convert the current frame target segmentation image into a binary image, mark the connected regions of the binary image, and correct the segmentation image according to the feature information after fusion processing to obtain the final current frame target segmentation image.
[0096] In one possible implementation, converting the current frame target segmentation image into a binary image and marking the connected regions of the binary image includes:
[0097] Obtain the pixels in the target segmentation image of the current frame and compare their radiance intensity with a preset intensity threshold;
[0098] If the radiation intensity is less than the threshold, the pixel gray value is set to 0; if it is greater than the threshold, it is set to 255, and a binary image is generated.
[0099] Connectivity analysis is used to number the independent regions in the binary image and extract the geometric attributes of each region, including contour, area, and centroid.
[0100] For example, the 8-neighborhood algorithm is used to number independent regions and extract geometric attributes such as area, centroid, and roundness; segmentation correction: noise is filtered based on arc feature information (such as temperature > 1000℃, area > 50 pixels, and motion speed < 10 pixels / frame), and small holes are filled through morphological opening and closing operations.
[0101] It should be noted that connected component labeling involves scanning the image and marking adjacent non-zero pixels as the same region. The accuracy of the segmentation results is improved by refining the segmented image. This refinement process includes noise removal and hole filling to ensure the integrity and accuracy of the segmentation results.
[0102] S500: Determine the valid arc fault based on the final target segmentation image of the current frame.
[0103] Specifically, the fault determination rules are as follows:
[0104] An electric arc region was detected for three consecutive frames and its area continued to increase; the average temperature of the electric arc region was >1200℃ and its trajectory conformed to the discharge law; and the determination was made in combination with the equipment operating parameters (such as current >1.5 times the rated value).
[0105] For example, in a 220kV substation application, the method achieved an accuracy of 96.3% in detecting initial arcs (area <100 pixels) and a false alarm rate of <3.2%, which is 21.5% higher than that of single-frame U-Net.
[0106] like Figure 6 As shown, a second aspect of the present invention provides a multi-frame infrared arc detection device based on a dual-stream network, comprising:
[0107] The image acquisition unit 100 is used to acquire multiple consecutive arc infrared images at preset millisecond time intervals, and input each infrared image sequentially into a pre-trained dual-stream network model.
[0108] The feature processing unit 200 is used to extract the motion features and appearance features of the electric arc from the current frame of the electric arc infrared image through the dual-stream network model, enhance the appearance features through motion-guided enhancement, and perform multi-scale progressive fusion processing on the motion features and the enhanced appearance features.
[0109] The segmentation optimization unit 300 is used to fuse high-level semantics into high-resolution features based on the differences in resolution and semantic information of features at different stages of the dual-stream network model, and to obtain the target segmentation image of the current frame corresponding to the current frame arc infrared image.
[0110] The image correction unit 400 is used to convert the current frame target segmentation image into a binary image, mark the connected regions of the binary image, and correct the segmentation image according to the feature information after fusion processing to obtain the final current frame target segmentation image.
[0111] Output unit 500 is used to determine valid arc faults based on the target segmentation image of the current frame.
[0112] like Figure 7 As shown, a third aspect of the present invention provides a terminal device, including an interactive device, an input / output (I / O) interface, a processor, and a memory, wherein the memory stores program instructions;
[0113] The interactive device is used to acquire user input commands;
[0114] The processor is used to execute program instructions stored in the memory to perform the multi-frame infrared arc detection method based on a dual-stream network as described above.
[0115] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the multi-frame infrared arc detection method based on a dual-stream network as described above.
[0116] Compared with existing technologies, the embodiments of the present invention fuse multi-frame spatiotemporal features through a dual-stream network, converting target detection into image segmentation. Combined with motion guidance and multi-scale fusion strategies, it effectively overcomes the false alarm and missed detection problems of single-frame detection, and improves the robustness and real-time performance of arc detection in complex industrial scenarios.
[0117] Those skilled in the art will realize that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A multi-frame infrared arc detection method based on a dual-stream network, characterized in that, The method includes at least the following steps: Multiple consecutive arc infrared images are acquired at preset millisecond time intervals, and each infrared image is sequentially input into a pre-trained dual-stream network model. The two-stream network model extracts the motion and appearance features of the electric arc from the current frame of the electric arc infrared image, enhances the appearance features through motion-guided enhancement, and performs multi-scale progressive fusion processing on the motion features and the enhanced appearance features. Based on the differences in resolution and semantic information of features at different stages of the dual-stream network model, high-level semantics are fused into high-resolution features, and the target segmentation image of the current frame corresponding to the current frame arc infrared image is obtained. The current frame target segmentation image is converted into a binary image, and the connected regions of the binary image are marked. The segmentation image is then corrected based on the feature information after fusion processing to obtain the final current frame target segmentation image. Valid arc faults are determined based on the final target segmentation image of the current frame.
2. The method according to claim 1, characterized in that, Before acquiring multiple consecutive arc infrared images at preset millisecond time intervals and sequentially inputting each infrared image into a pre-trained dual-stream network model, the method further includes: Create a two-stream network model; The equipment setup was verified and clear arc images were captured through preliminary experiments. Preprocessing of the infrared image training set yields preprocessed training images; Obtain training target segmentation image samples corresponding to each preprocessed training image; Feature data of the preprocessed training images are extracted using the dual-stream network model, and the model is trained using the segmented image samples of the training target as the output reference until the parameters converge.
3. The method according to claim 1 or 2, characterized in that, The architecture of the dual-stream network model is constructed by enhancing the Swin-Unet deep learning segmentation network through the introduction of a parallel dual-stream architecture; wherein, The architecture of the dual-stream network model includes a transformer architecture and parallel sub-networks with U-shaped feature extraction, thereby promoting feature fusion and utilizing a cross-attention mechanism to continuously exchange modules between modalities. Within the architecture of the dual-stream network model, the cross-transformer module uses deformable sampling to calculate the structural relationship between the two modalities, thereby reshaping the structural information of one modality to align with the corresponding structures of the two modalities within the same local window of the Swing Transformer. Through the interaction of the two modalities, while maintaining the original feature distribution of the modalities, it can enhance relevant segmentation features and reduce the influence of irrelevant features.
4. The method according to claim 1, characterized in that, The extraction of arc motion and appearance features from the current frame arc infrared image using the dual-stream network model includes: Dynamic features, i.e. motion features, between multiple frames can be extracted using optical flow networks or 3D convolution, including the displacement vector of the electric arc, flicker frequency, and rate of morphological change. Static features, or appearance features, of a single frame of infrared image are extracted using a convolutional neural network, including the shape, temperature gradient, and texture of high-temperature areas.
5. The method according to claim 4, characterized in that, Enhance appearance features through motion-guided enhancement methods, including: An attention mask is generated using the motion features of the time branch, including a saliency map of the dynamic region; The appearance features are weighted by a multiplicative attention mechanism to suppress static interference by weighting regions related to electric arc motion. Furthermore, based on a cross-attention mechanism, the similarity between the motion features and surrounding window features is calculated using semantic information from the motion information, thereby guiding the appearance features to learn semantics.
6. The method according to claim 5, characterized in that, The multi-scale progressive fusion processing of motion features and enhanced appearance features includes: Shallow high-resolution features and deep low-resolution features are fused in layers through skip connections and upsampling to generate a composite feature map that includes detailed localization and global semantics.
7. The method according to claim 1, characterized in that, The step of converting the current frame target segmentation image into a binary image and marking the connected regions of the binary image includes: Obtain the pixels in the target segmentation image of the current frame and compare their radiance intensity with a preset intensity threshold; If the radiation intensity is less than the threshold, the pixel gray value is set to 0; if it is greater than the threshold, it is set to 255, and a binary image is generated. Connectivity analysis is used to number the independent regions in the binary image and extract the geometric attributes of each region, including contour, area, and centroid.
8. A multi-frame infrared arc detection device based on a dual-stream network, characterized in that, include: The image acquisition unit is used to acquire multiple consecutive arc infrared images at preset millisecond time intervals, and input each infrared image sequentially into a pre-trained dual-stream network model. The feature processing unit is used to extract the motion features and appearance features of the electric arc from the current frame of the electric arc infrared image through the dual-stream network model, enhance the appearance features through motion-guided enhancement, and perform multi-scale progressive fusion processing on the motion features and the enhanced appearance features. The segmentation optimization unit is used to fuse high-level semantics into high-resolution features based on the differences in resolution and semantic information of features at different stages of the dual-stream network model, and to obtain the target segmentation image of the current frame corresponding to the current frame arc infrared image. The image correction unit is used to convert the current frame target segmentation image into a binary image, mark the connected regions of the binary image, and correct the segmentation image according to the feature information after fusion processing to obtain the final current frame target segmentation image. The output unit is used to determine valid arc faults based on the target segmentation image of the current frame.
9. A terminal device, characterized in that, It includes an interactive device, an input / output (I / O) interface, a processor, and a memory, wherein program instructions are stored in the memory; The interactive device is used to acquire user input commands; The processor is used to execute program instructions stored in the memory to perform the multi-frame infrared arc detection method based on a dual-stream network as described in any one of claims 1-7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the multi-frame infrared arc detection method based on a dual-stream network as described in any one of claims 1-7.